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Abstract 


This  document  presents  a  modeling  framework  for  estimating  the  cost  and  benefits  of  various 
team  structures  in  Command  and  Control  (C2).  The  proposed  strategy  combines  task  analysis  and 
data-driven  modeling  as  a  means  to  deal  with  effective  team  design  in  military  C2.  A  hierarchical 
task  analysis  (HTA)  identified  the  set  of  subtasks  associated  with  different  components  of  a 
simulated  C2  task.  For  each  team  structure  considered,  we  performed  a  task-to-agent  mapping 
based  on  the  HTA  results  in  order  to  identify  interpersonal  task  dependencies  and  identify  the 
activities  associated  with  each  agent  (taskwork,  teamwork  and  interaction  with  tools).  A 
mathematical  model  is  then  developed  to  quantify  the  impact  of  each  element  of  that  mapping  on 
individual  workload.  Furthermore,  the  model  specifies  the  effects  of  workload  on  team 
performance  as  a  function  of  interpersonal  dependencies.  The  calibration  of  the  model  is  based  on 
an  empirical  study  that  tests  two  team  structures.  We  then  describe  a  tool  based  on  this  model  and 
test  its  potential  to  estimate  the  effectiveness  of  other  team  structures  in  order  to  identify  the 
optimal  team  design  for  the  simulated  C2  task.  Finally,  future  work  for  expanding  this  team 
design  tool  to  different  C2  domains  is  discussed. 


Resume 


Ce  document  presente  un  cadre  theorique  de  modelisation  permettant  V  estimation  des  couts  et 
benefices  de  differentes  structures  d’equipe  en  Commandement  et  Controle  (C2).  La  strategie 
utilisee  combine  une  analyse  de  tache  a  une  modelisation  basee  sur  les  donnees  afin  d’influencer 
le  developpement  et  la  formation  d’equipes  pour  des  situations  de  C2  militaire.  Une  analyse 
hierarchique  des  taches  a  contribue  a  V identification  d’un  ensemble  de  sous-taches  associees  aux 
differentes  composantes  d’une  tache  de  C2  simulee.  Pour  chaque  structure  d’equipe,  une 
association  tache-a-agent  basee  sur  les  resultats  de  T analyse  hierarchique  est  effectuee  afin 
d’ identifier  les  dependances  interpersonnelles  dans  V  execution  des  differentes  taches  et  les 
activites  pour  chaque  agent  relie  aux  taches  a  executer,  au  fait  de  faire  partie  d’une  equipe  et  au 
fait  d’interagir  avec  des  outils.  Un  modele  mathematique  est  alors  developpe  pour  quantifier 
l’impact  de  chacun  de  ces  elements  sur  la  charge  de  travail  de  chaque  agent.  De  plus,  le  modele 
specifie  les  effets  de  la  charge  de  travail  sur  la  performance  de  V  equipe  en  fonction  des 
dependances  interpersonnelles.  L’etalonnage  du  modele  est  base  sur  une  etude  empirique  qui 
mesure  la  performance  de  deux  equipes  adoptant  des  structures  organisationnelles  differentes.  Le 
modele  mathematique  etalonne  est  alors  utilise  pour  tester  son  potentiel  a  e  valuer  l’efficacite 
d’autres  structures  d’equipe  afin  d’ identifier  la  structure  d’equipe  optimale  pour  1’ execution  de  la 
tache  simulee  de  C2.  Finalement,  une  discussion  concemant  les  travaux  futurs  necessaires  a 
1’ expansion  de  cet  outil  de  design  d’equipe  aux  differents  domaines  de  C2  est  presentee. 
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Executive  summary 


Developing  a  framework  for  estimating  the  costs/benefits  of 
teamwork  in  C2: 

Richard  Breton;  Daniel  Lafond;  Sebastien  Tremblay;  Robert  Rousseau;  DRDC 
Valcartier  TR  2010-225;  Defence  R&D  Canada  -  Valcartier;  September  2011. 

Introduction:  This  document  presents  a  modeling  framework  for  estimating  the  cost  and  benefits 
of  various  team  structures  in  Command  and  Control  (C2).  The  proposed  strategy  combines  task 
analysis  and  data-driven  modeling  as  a  means  to  deal  with  effective  team  design  in  military  C2. 

Results:  A  hierarchical  task  analysis  (HTA)  identified  the  set  of  subtasks  associated  with 
different  components  of  a  simulated  C2  task.  A  task-to-agent  mapping  based  on  the  HTA  results 
identified  interpersonal  task  dependencies  and  the  activities  associated  with  each  agent  (taskwork, 
teamwork  and  interaction  with  tools)  for  each  team  structure  considered.  A  mathematical  model, 
calibrated  from  an  empirical  study  involving  two  team  structures,  quantified  the  impact  of  each 
element  of  that  mapping  on  individual  workload.  Furthermore,  the  model  specified  the  effects  of 
workload  on  team  performance  as  a  function  of  interpersonal  dependencies. 

One  major  impact  of  this  project  was  the  demonstration  of  the  relationship  between  key 
determinants  of  teamwork  and  team  effectiveness  that  varied  as  a  function  of  team  structure.  This 
demonstration  came  from  the  comparison  of  the  coefficients  found  in  two  multiple  regression 
models  (for  multifunctional  and  functional  teams)  relating  team  performance  to  a  number  of 
predictors  of  team  effectiveness. 

Also,  key  similarities  and  differences  were  observed,  suggesting  that  some  aspects  of  team 
functioning  can  markedly  change  as  a  function  of  team  structure.  The  findings  presented  in  this 
report  suggested  that  teamwork  requirements  and  the  importance  of  various  team  factors  in 
general  may  vary  as  a  function  of  team  structure.  Obviously,  communication  and  coordination  are 
key  components  of  teamwork,  but  their  relative  importance  is  affected  by  the  manner  in  which  the 
team  is  configured.  For  instance,  results  showed  that  communication  was  positively  correlated 
with  team  performance  in  functional  teams,  but  negatively  correlated  in  multi-functional  ones. 

Finally,  the  mathematical  model  displayed  an  impressive  capacity  to  replicate  the  results  observed 
from  the  20  three-person  teams.  From  this  model,  it  was  possible  to  predict  the  performance  of 
four  other  fictional  teams. 

Significance:  Teamwork  is  often  essential  in  complex  and  dynamic  environments  such  as  C2.  In 
the  domains  of  military  operations  and  homeland  security,  there  is  a  growing  interest  in  team 
research,  with  the  intention  to  improve  efficacy  through  the  use  of  networking.  There  are 
important  issues  with  regards  to  inter-team  operability  and  intra-team  optimization  of 
performance.  This  project  provided  interesting  findings  in  regard  to  these  C2  issues. 

First,  since  different  C2  team  structures  may  rely  upon  different  team  processes,  it  would  seem 
inadequate  for  future  research  to  employ  a  general  model  of  team  performance  that  uses  a  fixed 
set  of  standard  team  processes  as  predictor  variables.  The  role  of  these  processes  is  not  the  same 
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from  one  team  structure  to  another,  and  so  such  processes  are  an  insufficient  way  to  predict  the 
performance  of  different  team  structures. 


Second,  this  project  defined  the  level  of  individual  workload  as  a  key  predictor  of  team 
performance.  The  correlation  between  the  level  of  workload  perceived  by  the  participants  and  the 
results  from  the  model  was  astonishing.  This  remarkable  result  suggested  that  the  workload 
metric  based  on  the  task-to-agent  mapping  represents  accurately  the  actual  workload  of 
individuals. 

Future  plans:  The  possibility  to  use  such  models  in  order  to  identify  the  optimal  type  of  team 
structure  and  the  optimal  number  of  team  members  is  promising.  To  make  it  possible,  three 
phases  should  follow  the  work  initiated  in  this  project: 

Phase  1 :  The  comparison  of  the  results  predicted  by  the  model  with  the  four  different  team 
structures  (Six-Person  multifunctional;  four-Person  functional;  Hybrid;  and  Alternate  functional) 
with  the  performance  of  teams  using  these  different  structures.  This  would  provide  a  test  of  the 
specific  assumptions  proposed  in  the  model  and  help  improve  its  reliability. 

Phase  2:  The  replication  of  the  results  presented  in  this  report  with  a  different  microworld  in 
order  to  generalize  the  findings  and  increase  the  ecological  validity  of  the  approach. 

Phase  3:  The  validation  of  the  capacity  of  the  model  to  assist  team  design  by  testing  its 
predictions  of  performance  for  real  C2  teams  in  the  ecological  context  of  a  rapid  response 
planning  exercise  or  integrated  C2  experimentation. 
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Sommaire 


Developing  a  framework  for  estimating  the  costs/benefits  of 
teamwork  in  C2: 

Richard  Breton;  Daniel  Lafond;  Sebastien  Tremblay;  Robert  Rousseau;  DRDC 
Valcartier  TR  2010-225;  R  &  D  pour  la  defense  Canada  -  Valcartier;  Septembre 
2011. 

Introduction:  Ce  document  presente  un  cadre  theorique  de  modelisation  pour  1’ estimation  des 
couts  et  benefices  relies  a  differentes  structures  d’equipe  de  C2.  La  strategie  proposee  combine 
une  analyse  de  tache  a  une  modelisation  basee  sur  les  donnees  afin  de  supporter  le  besoin  de 
design  approprie  d’equipes  pour  les  situations  militaires  de  C2. 

Resultats:  Une  analyse  de  tache  hierarchique  a  permis  1’ identification  d’un  ensemble  de  sous- 
taches  associees  aux  differentes  composantes  de  la  tache  simulee  de  C2.  Une  association  tache- 
agent  basee  sur  les  resultats  de  cette  analyse  a  demontre  des  dependances  interpersonnelles  entre 
les  taches  et  les  activites  associees  aux  differents  agents  (activites  par  rapport  a  la  tache,  a 
T  equipe  et  aux  interactions  avec  les  outils).  Un  modele  mathematique,  etalonne  a  partir  d’une 
etude  empirique  impliquant  deux  structures  d’equipe,  a  permis  la  quantification  de  1’ impact  de 
chaque  element  de  ces  associations  sur  la  charge  de  travail  mental  de  chaque  agent.  De  plus,  le 
modele  specifie  les  effets  de  cette  charge  de  travail  sur  le  niveau  de  performance  de  T  equipe  en 
fonction  des  relations  interpersonnelles. 

Un  impact  majeur  de  ce  projet  conceme  la  demonstration  de  la  relation  entre  des  concepts 
determinants  du  travail  en  equipe  et  l’efficacite  de  T  equipe  variant  en  fonction  de  la  structure  de 
T  equipe.  Cette  demonstration  provient  de  la  comparaison  des  coefficients  trouves  a  partir  de  deux 
regressions  multiples  (pour  1’ equipe  fonctionnelle  et  1’ equipe  multifonctionnelle)  reliant  alors  la 
performance  de  chacune  de  ces  deux  equipes  a  un  nombre  de  parametres  de  prediction  pour 
l’efficacite  de  1’ equipe. 

Egalement,  des  differences  et  similarites  importantes  entre  les  deux  structures  d’equipe  ont  ete 
observees,  suggerant  que  certains  aspects  du  fonctionnement  de  1’ equipe  peuvent  changer  de 
fagon  significative  en  fonction  de  la  structure  d’equipe.  Les  resultats  demontres  dans  ce  rapport 
suggerent  que  les  besoins  du  travail  en  equipe  et  l’importance  de  differents  facteurs  d’equipe  en 
general  varient  done  en  fonction  de  la  structure  de  1’ equipe.  Evidemment,  la  communication  et  la 
coordination  sont  deux  composantes  importantes  du  travail  en  equipe.  Par  contre,  leur  importance 
relative  est  affectee  par  la  fagon  dont  1’ equipe  est  configuree.  Par  exemple,  les  resultats  ont 
demontre  que  la  communication  etait  positivement  correlee  avec  la  performance  pour  1’ equipe 
fonctionnelle,  mais  negativement  correlee  pour  1’ equipe  multifonctionnelle. 

Finalement,  le  modele  mathematique  a  demontre  une  capacite  impressionnante  a  reproduire  les 
resultats  observes  lors  de  1’ etude  empirique.  Done,  a  partir  de  ce  modele,  il  a  ete  possible  de 
simuler  et  predire  la  performance  de  quatre  autres  types  de  structures  d’equipe. 

Importance:  Le  travail  en  equipe  est  souvent  essentiel  dans  des  environnements  complexes  et 
dynamiques  tels  que  le  C2.  Dans  les  domaines  des  operations  militaires  et  de  la  securite 
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interieure,  il  y  a  un  interet  croissant  pour  la  recherche  concernant  les  equipes  avec  l’intention 
d’ameliorer  leur  efficacite  par  le  reseautage.  Plusieurs  questions  concernent  1’ optimisation  de 
l’operabilite  inter-equipe  et  intra-equipe.  Ce  projet  amene  des  decouvertes  interessantes 
concernant  ces  questions. 

Premierement,  puisque  differentes  structures  d’equipe  de  C2  impliquent  differents  processus 
d’equipe,  il  semblerait  inadequat  que  les  recherches  futures  se  basent  sur  un  modele  generique 
considerant  un  ensemble  fixe  d’une  structure  a  V autre  de  processus  d’equipe  pour  predire  la 
performance.  Le  role  ne  semblant  pas  etre  le  meme  d’une  equipe  a  1’ autre,  un  ensemble  fixe  ne 
peut  done  pas  permettre  de  predire  efficacement  la  performance  en  fonction  des  differentes 
structures  d’equipe. 

Deuxiemement,  ce  projet  permet  de  definir  le  niveau  de  charge  de  travail  mental  individuel 
comme  un  element  cle  pour  predire  la  performance  de  1’ equipe.  La  correlation  entre  le  niveau  de 
charge  de  travail  mental  pergue  par  les  participants  et  le  resultat  de  la  modelisation  (charge 
predite)  est  etonnante.  Ce  resultat  remarquable  suggere  que  la  metrique  utilisee  dans  le  modele 
pour  la  charge  de  travail  mental  qui  est  basee  sur  1’ association  entre  les  taches  et  les  agents 
represente  tres  precisement  la  charge  de  travail  mental  reelle  des  participants. 

Perspectives:  La  possibility  d’utiliser  ce  modele  mathematique  pour  identifier,  en  fonction  du 
contexte,  la  structure  optimale  d’une  equipe  ainsi  que  le  nombre  optimal  d’ agents  dans  1’ equipe 
est  prometteur.  Pour  rendre  cela  possible,  certaines  phases  sont  toutefois  requises: 

Phase  1 :  La  comparaison  des  resultats  predits  par  le  modele  pour  les  quatre  structures  d’equipe 
(six-personnes  multifonctionnelle;  quatre-personnes  fonctionnelle;  hybride;  et  fonctionnelle 
alternative)  avec  la  performance  reelle  d’ equipes  basees  selon  ses  structures.  Cela  permettrait  de 
tester  les  hypotheses  proposees  par  le  modele  et  d’augmenter  sa  fiabilite. 

Phase  2:  La  reproduction  des  resultats  presentes  dans  ce  rapport  avec  un  micro-monde  different. 
Cela  permettrait  de  generaliser  les  resultats  et  d’augmenter  la  validite  ecologique  de  l’approche. 

Phase  3:  La  validation  de  la  capacite  du  modele  d’assister  le  design  d’equipe  en  testant  ces 
predictions  de  la  performance  pour  des  equipes  reelles  de  C2  dans  un  contexte  ecologique  de 
processus  de  planification  rapide  ou  des  experimentations  sur  le  C2  integre. 
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1  Introduction 


Teamwork  is  often  essential  in  complex  and  dynamic  environments  such  as  command  and  control 
(C2).  In  the  domains  of  military  operations  and  homeland  security,  there  is  a  growing  interest  in 
team  research,  with  the  intention  to  improve  efficacy  through  the  use  of  networking.  There  are 
important  issues  with  regards  to  inter-team  operability  and  intra-team  optimization  of 
performance.  At  the  tactical  level,  the  context  of  operations  is  often  one  of  high  constraints  in 
terms  of  time  pressure  and  uncertainty.  Small  teams  of  military  units,  security  or  emergency 
workers  are  often  confronted  with  such  situations  in  which  they  must  work  together  in  order  to 
achieve  safety-critical  goals  and  mission  success.  In  C2  situations,  the  expertise  and  resources 
required  for  the  successful  achievement  of  the  mission  most  often  go  beyond  the  capability  of  a 
single  individual.  However,  the  addition  of  people  in  the  execution  of  the  tasks  represents  in  itself 
an  element  of  complexity.  In  the  current  project,  the  goal  is  to  develop  a  methodology  that  will 
enable  users  to  estimate  the  cost  and  benefits  of  teamwork  in  tactical  C2  for  intra-team  activities 
and  identify  team  structures  that  support  optimal  team  performance. 

The  structure  of  a  team  may  range  from  rigid,  with  clearly  defined  roles  and  a  hierarchical  chain- 
of-command,  to  flexible,  where  individuals  all  have  similar  capabilities,  tasks  are  allocated 
flexibly,  and  decisions  are  made  jointly  by  consensus.  Different  team  structures  may  involve 
particular  benefits  resulting  from  the  leverage  achieved  by  combining  individual  capabilities  in 
different  ways.  These  structures  impose  teamwork  requirements.  Coordination  and 
communication  among  team  members  come  with  a  cost  in  performance.  Some  team  structures 
may  exacerbate  teamwork  requirements  to  the  point  that  no  benefits  are  observed,  while  others 
can  maximize  the  benefits  without  entailing  extensive  interaction  costs.  Besides  the  context  of 
operations  and  environmental  constraints,  the  effectiveness  of  different  team  structures  depends 
on  this  interplay  between  costs  and  benefits.  Since  this  interplay  is  not  well  understood,  the  aim 
of  the  present  work  is  to  develop  a  model  of  the  effects  of  the  organizational  structure  on  team 
performance. 

In  the  following  sections  of  this  report,  we  will  provide  a  background  on  relevant  research 
(Section  2)  on  C2,  team  cognition  and  team  structures  and  report  different  steps  in  the 
development  of  a  methodology  for  estimating  the  costs/benefits  of  teamwork  in  C2  (Sections  3- 
5).  Our  methodology  can  be  described  in  three  parts: 

♦  A  data-driven  analysis  (reported  in  Section  3)  focused  on  the  experimental  study  of 
team  effectiveness  as  a  function  of  team  structure. 

♦  A  task-driven  analysis  (reported  in  Section  4)  focused  on  a  hierarchical 
decomposition  of  subtasks.  The  analysis  is  complemented  by  an  assessment  of  tools, 
information,  triggering  events  and  actions  associated  with  each  subtask. 

♦  An  integration  of  the  data-driven  and  task-driven  approach  (Section  5),  involving  a 
task-to-agent  mapping  that  characterizes  the  teamwork,  taskwork  and  tool 
interactions  for  each  agent  and  task  dependency.  This  mapping  forms  the  basis  for  a 
mathematical  model  calibrated  on  the  experimental  data  and  used  to  predict  the 
effectiveness  of  different  team  structures. 
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2  Background 


2.1  Command  and  control 

According  to  Curts  and  Campbell  [1],  C2  can  be  defined  as  “the  exercise  of  direction  by  a 
designated  leadership  over  resources  in  the  accomplishment  of  a  mission”.  Pigeau  and  McCann 
[2]  define  command  as  “the  creative  expression  of  human  will  necessary  to  accomplish  the 
mission”  and  control  as  “those  structures  and  processes  devised  by  command  to  enable  it  and  to 
manage  risk”.  While  C2  is  predominantly  associated  with  military  operations,  it  also  applies  to 
many  non-military  activities  such  as  fire  fighting,  air-traffic  control,  nuclear  power  plant 
operations,  hospital  emergency  rooms,  and  incident  command  in  crisis  management  (e.g.,  rescue 
operations  following  a  disaster). 

Military  C2  can  be  defined  as  the  control  of  spatially  distributed  assets  (e.g.,  sensors)  as 
efficiently  as  possible  in  order  to  achieve  tactical  goals  [3]  and  consists  of: 

1.  information  gathering  and  situation  assessment; 

2.  identification  or  classification  of  the  situation  based  on  situations  typically  encountered; 

3.  proposal  of  a  solution; 

4.  evaluation  and  refinement  of  the  solution  by  projection  of  consequences  and  events  into  the 
near  future;  and 

5.  execution  of  the  response  and  continued  monitoring  of  the  situation  to  ensure  it  evolves  as 
desired  [4]. 

C2  can  take  place  at  three  distinct  levels  each  having  its  own  contextual  characteristics: 

♦  A  strategic  context:  Decision  making  at  the  strategic  level  is  about  forming  a 
comprehensive  view  of  the  situation,  reasoning  about  its  meaning,  finding  possible 
outcomes  and  then  choosing  what  policy  to  adopt.  It  requires  considering  trends, 
long  term  effects  and  global  outcomes.  Strategic  planning  focuses  on  the  overall 
context,  not  the  specifics.  In  sum,  it  is  about  defining  a  general  strategy  to  reach  long 
term  objectives.  Collaborative  decision  making  at  the  strategic  level  is  removed  from 
the  action  and  generally  occurs  under  low  levels  of  time -pressure. 

♦  An  operational  context:  Operational  decision-making  requires  more  detailed 
planning  based  on  the  specifics  of  the  situation.  This  level  aims  to  coordinate  tactics 
in  order  to  reach  strategic  goals.  Objectives  are  oriented  toward  a  specific  situation. 
Here,  the  time  window  is  smaller  than  at  the  strategic  level  and  the  team  is  closer  to 
the  action  (but  still  removed  from  it).  Information  management  is  an  important 
aspect  of  operational  decision-making,  though  the  scope  of  situation  is  more 
restrained  than  at  the  strategic  level. 
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♦  A  tactical  context:  A  tactical  team  is  in  the  middle  of  the  action  and  is  focused  on 
task  execution.  Tactical  decision-making  is  therefore  focused  on  the  immediate 
situation  and  on  the  technical  aspects  of  the  actions  to  perform.  Time  is  the  essence 
and  decisions  must  be  made  without  complex  reasoning  and  deliberation.  There  is 
little  opportunity  for  extensive  information  gathering  and  management. 
Collaboration  in  this  context  is  mainly  about  sharing  awareness,  mutual 
coordination,  mutual  support  and  the  application  of  team  training.  This  is  the 
decision  making  level  being  focused  on  in  the  present  report. 

Figure  1  shows  that  the  tactical,  operation  and  strategic  levels  can  be  place  on  a  continuum  in 
which  levels  of  decision-making,  task  execution,  time-pressure  and  information  load  either 
increase  or  decrease.  The  strategic  level  is  mainly  about  taking  decisions  based  on  the  best 
information  possible.  At  the  other  extreme,  tactical  decision-making  must  be  quick  and  mainly 
leads  to  task  execution  rather  than  information  management. 


Figure  1:  Levels  of  decision-making. 

In  the  context  of  military  operations,  C2  is  commonly  described  as  the  cyclic  and  continuous 
operation  of  four  functions  (see  Figure  2):  Observe-Orient-Decide-Act  referred  to  as  the  OODA 
loop  [5]. 


Orient 


Decide 


Figure  2:  The  classical  version  of  the  OODA  loop  model. 
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♦  Observe  is  to  collect  information  about  the  environment  (enemy  position,  status, 
intent,  friendly  forces  position).  The  effectiveness  of  this  phase  relies  on  the  amount, 
quality,  appropriateness,  and  timeliness  of  information. 

♦  Orient  is  to  develop  knowledge  and  judgments  about  the  situation.  This  is  where  the 
awareness  of  the  situation  is  developed  and  is  based  on  cognitive  processes  of 
knowledge  development.  The  ability  to  convert  data  into  information  that  develops 
knowledge  that  supports  understanding  is  what  is  taking  place  in  this  cognitive 
process. 

♦  Decide  is  the  course  of  action  taken  that  develops  in  some  plan.  The  ability  to  make 
sound  and  timely  decisions  is  an  important  part  of  effective  command  and  control. 

The  ability  to  make  good  decisions  faster  considering  the  uncertainties  provides  an 
advantage. 

♦  Act  is  taken  to  convey  the  commander’s  intent  through  orders.  This  may  be  through 
detailed  orders  or  by  passing  along  high  level  objectives.  This  also  includes 
monitoring  the  execution  of  operations  that  required  information  in  the  “Observe” 
phase  of  command  and  control. 

The  OODA  can  be  understood  as  a  simple  control  system  [6].  It  stresses  the  importance  of  two 
critical  factors  in  the  environment,  time  constraints  and  information  uncertainty,  on  the  execution 
of  the  decision  cycle.  In  order  to  deal  with  the  time  constraints,  the  phases  of  the  loop  must  be 
executed  as  quickly  as  possible.  To  reduce  the  information  uncertainty,  they  must  be  performed 
accurately.  The  antagonistic  relationship  between  these  two  factors  is  commonly  referred  to  as  the 
speed-accuracy  trade-off  [7]. 

Decision-making  at  the  strategic,  operational  or  tactical  level  can  be  described  by  the  OODA 
phases.  However,  these  three  levels  of  decision-making  are  characterized  by  different  levels  of 
time  pressure  and  the  amount  of  information  that  can  be  processed  in  order  to  cope  with  the 
situation  uncertainty. 

2.1.1  Teamwork  in  C2 

Historically,  C2  has  been  seen  as  a  hierarchical  process  of  commanders  directing  their 
subordinates  on  the  battlefield  (though  generalized  command  and  control  also  has  many  non¬ 
military  applications  as  well).  However,  more  recently  there  has  been  an  increasing  appreciation 
of  the  distributed  nature  of  information  collection.  Often,  decisions  must  be  coordinated  laterally 
between  multiple  units  involved,  and  occasionally  there  is  a  need  to  push  decisions  further  down 
to  smaller  units  closer  to  the  battles,  which  have  a  better  sense  of  tactical  opportunities  and 
consequences  of  actions.  Hierarchical  command  is  now  even  viewed  by  some  as  inflexible  and 
sub-optimal. 

Teamwork  is  often  associated  with  C2.  Many  definitions  of  teamwork  have  been  proposed,  but 
one  common  element  is  that  teamwork  is  about  several  individuals  working  toward  a  shared  goal 
(see  [8]).  For  instance,  McIntyre  and  Salas  [9]  describe  a  team  as  a  “set  of  two  or  more 
individuals  who  interact  interdependently  and  adaptively  to  achieve  specified,  shared  and  valued 
objectives”.  Each  team  member  thus  has  to  perform  some  tasks  associated  with  his  or  her  role 
within  the  team.  In  order  to  make  an  optimal  decision  information  has  to  be  effectively  merged 
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among  team  members.  Salas,  Bowers,  and  Cannon-Bowers  [10]  assert  that  although  significant 
advances  have  been  made  in  understanding  team  processes,  much  remains  to  be  done.  Indeed,  a 
lack  of  consensus  regarding  the  delimitation  and  conceptualization  of  processes  and  behaviours 
critical  for  team  effectiveness  is  evident  in  the  literature  on  teamwork  (see  [11;  12]). 

While  some  principles  of  teamwork  may  turn  out  to  be  universal,  there  is  a  growing  consensus 
that  factors  of  team  effectiveness  are  context-dependent.  This  idea  also  relates  to  the  basic  tenet 
of  the  study  of  situated  cognition:  “Human  intellectual  resources  engage  with,  and  cannot  usefully 
be  seen  as  separate  from,  the  physical  resources  in  the  world  in  task  performance  [13]”. 
Traditional  experimental  studies  tend  to  ignore  the  situated  and  organizational  features  of  work. 
People  use  a  large  range  of  resources  in  their  work  environment.  These  resources  and  tools  are 
typically  removed  in  laboratory  settings  so  that  they  do  not  interfere  with  the  “controlled 
variables”.  These  resources  and  the  interactions  between  them  are  critical  for  understanding  much 
of  our  behaviour  in  actual  work  settings  [14]. 

Consequently,  there  is  a  growing  need  to  understand  teamwork  and  the  processes  or  factors  that 
affect  the  team  performance.  These  processes  or  factors  are  context-dependent.  Then,  it  is 
important  to  develop  an  approach  to  support  the  optimal  team  structure  based  on  the  context  in 
which  teamwork  occurs. 

2.1 .2  Fields  of  study  of  teamwork 

The  term  “teamwork”  can  be  found  in  the  management,  psychology  and  human  factors  literature, 
whereas  “collaborative  work”  is  most  often  employed  in  the  technology-related  literature. 
Teamwork  is  an  active  research  topic  in  many  fields,  particularly  in  distributed  artificial 
intelligence  (including  multiagent  systems  and  computational  organizational  theory)  and  in  the 
field  of  dynamic  decision  making. 

Distributed  artificial  intelligence  (DAI)  refers  to  the  design  and  application  of  multiagent  systems, 
in  which  interacting  intelligent  agents  pursue  a  set  of  goals  or  perform  a  set  of  tasks.  Agent 
operations  can  be  affected  by  interactions  with  other  agents  and  humans.  The  main  interaction 
patterns  in  multiagent  systems  are  goal  and  task-oriented  coordination,  in  cooperative  and  in 
competitive  situations.  DATs  long-term  goal  is  to  develop  methods  enabling  agents  to  interact 
and  to  advance  the  understanding  of  interactions  among  intelligent  entities  whether  they  are 
computational  or  human.  Multiagent  systems  (MAS)  study  networked  systems  composed  of 
distributed  agents  (humans,  robots,  computational  problem  solvers)  that  can  deliberate,  predict, 
communicate,  and  cooperate  [15].  MAS  can  be  characterized  as  having: 

1 .  no  explicit  global  control; 

2.  distributed  resources,  expertise,  intelligence,  and  processing  capabilities; 

3.  an  open  environment  full  of  uncertainties;  and 

4.  an  emphasis  placed  on  social  agency  and  social  commitments. 

The  Computational  Organization  Theory  (COT)  uses  models  and  simulations  to  analyze 
organizations  and  their  processes,  adopting  concepts  from  DAI  and  from  the  work  of 
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organizational  theorists.  Advances  in  COT  are  mainly  achieved  through  the  use  of  multiagent 
modeling  techniques,  where  intelligent  agents  generally  cooperate  together  to  achieve  some 
collective  goal.  Agents  can  take  specific  social  roles,  possess  a  given  set  of  cognitive  capabilities, 
and  have  particular  tasks  to  accomplish.  COT  examines  how  organizations  composed  of  people, 
artificial  agents,  or  both  should  be  coordinated  and  how  tasks  should  be  distributed.  Some  of  the 
main  research  topics  in  this  area  are  organizational  design,  knowledge  sharing,  learning,  and 
adaptability.  Formal  concepts  and  specific  measures  of  organizational  design  are  taken  from 
research  in  the  areas  of  coordination  [16],  social  networks  [17]  and  distributed  control  [18]. 

Team  decision  making  generally  occurs  in  dynamic  task  environments.  Dynamic  decision-making 
research  investigates  how  people  control  dynamic,  complex,  real-world  systems  [19]. 

A  dynamic  system  is  one  in  which: 

1.  a  series  of  activities  are  required  to  reach/maintain  the  overall  goal; 

2.  activities  depend  on  the  outcome  of  previous  activities; 

3.  task  parameters  are  continuously  varying  in  response  to  changes;  and 

4.  tasks  are  accomplished  in  real  time. 

Dynamic  environments  involve  a  variety  of  cognitive  processes  such  as  monitoring,  recognition, 
causal  learning,  search,  planning,  judgment,  and  choice  [19].  The  ability  to  coordinate  these 
processes  while  performing  under  time  constraints  is  a  key  to  achieving  system  control.  The 
decision  makers  must  analyze  the  state  of  the  system  and  attempt  to  influence  existing  processes 
in  order  to  achieve  a  desired  state  [20].  This  type  of  environment  is  often  characterized  by 
changing  conditions,  time  pressure,  uncertainty  and  an  overwhelming  amount  of  information  to 
process.  A  multitude  of  stressors  may  thus  be  identified:  multiple  information  sources; 
incomplete,  conflicting  information;  rapidly  changing,  evolving  scenarios;  time  pressure;  high 
work/information  load;  and  threat  (see  [21]). 

2.2  Teamwork  and  cognition 

For  Cannon-Bowers,  Tannenbaum,  Salas,  and  Volpe  [22],  team  processes  refer  to  behavioral  and 
cognitive  procedures  required  to  perform  the  task  so  that  the  team  goal  can  be  attained. 
Comparatively  to  individual  work,  teamwork  involves  unique  behaviours,  such  as  coordination 
and  information  sharing.  In  that  context,  behaviours  specific  to  teamwork  are  critical,  since 
individual  behaviours  are  not  sufficient  for  team  effectiveness  [23].  As  pointed  out  by  Cannon- 
Bowers  and  Salas  [24],  teamwork  behaviours  are  more  than  just  individual  abilities  applied  to 
team  tasks.  In  fact,  because  they  cannot  be  easily  quantified-as  opposed,  for  instance,  to  team 
performance-team  behaviours  constitute  one  of  the  most  difficult  elements  to  capture. 
Consequently,  researchers  in  the  team  domain  have  struggled  to  identify  and  operationalize  the 
critical  behaviours  that  allow  a  team  to  work  effectively  [25]. 
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2.2.1  Teamwork  vs  taskwork 


A  distinction  is  often  made  between  taskwork  and  teamwork.  Individuals  will  have  two  main 
types  of  behaviour:  those  that  are  related  to  the  individual  tasks,  referred  to  as  taskwork;  and 
those  that  are  related  to  team  members  in  order  to  coordinate  tasks  and  share  information,  referred 
to  as  teamwork  [22].  Teamwork  is  about  working  toward  a  shared  goal  (e.g.,  [26]).  Those 
behaviours  of  team  members  that  engender  a  sharing  of  information  and  a  coordination  of 
activities  are  collectively  called  teamwork.  Teamwork  is  a  behavioural  construct;  it  is  a  composite 
of  behaviours  that  team  members  exhibit  to  support  each  other  in  their  tasks. 

2.2.2  Teamwork  behaviours 

A  long  list  of  teamwork  behaviours  are  identified  and  described  in  the  literature,  and  all  seem 
more  or  less  related  to  team  performance.  In  the  various  taxonomies  (e.g.,  [25;  27;  28])  and 
models  (e.g.,  [29;  30;  31])  that  have  been  put  forward,  teamwork  behaviours  most  often  include 
communication,  adaptability,  coordination,  performance  monitoring/feedback,  team  leadership, 
team  situation  awareness  and  decision  making,  conflict  resolution,  information  management,  and 
task  distribution. 

Salas  et  al.  [12]  point  out  that  owing  to  the  lack  of  consensus  among  taxonomies  and  models, 
there  is  a  need  to  reduce  the  number  of  factors  that  are  crucial  to  teamwork.  Based  on  an 
exhaustive  survey  of  the  literature,  they  propose  five  core  components  to  teamwork:  team 
leadership,  performance  monitoring,  backup  behaviour,  adaptability,  and  team  orientation  (that  is, 
the  inclination  to  consider  other  team  members’  input  and  behaviours  in  the  execution  of  team 
tasks  and  to  improve  individual  performance  by  doing  so).  While  acknowledging  that  other 
variables  can  affect  teamwork,  Salas  et  al.  argue  that  these  five  elements  are  the  ones  with  the 
greatest  impact  on  team  performance  and  that  are  identified  as  important  to  teamwork  in  a 
majority  of  taxonomies.  The  authors  also  suggest  that  the  successful  enactment  of  these 
components  requires  the  presence  of  three  coordination  mechanisms  between  team  members: 
shared  mental  models,  closed-loop  communication,  and  mutual  trust. 

Rousseau  et  al.  [11]  propose  a  comprehensive  framework  of  teamwork  behaviours  that  is 
structured  hierarchically.  The  authors  assert  that  teamwork  behaviours  hold  two  main  functions, 
regulation  of  team  performance  and  management  of  team  maintenance,  which  are  divided  into 
sub-categories  of  teamwork  dimensions.  While  management  of  team  concerns  personal  and  inter¬ 
personal  factors  (psychological  support  and  conflict  management),  regulation  of  team 
performance  integrates  teamwork  behaviours  found  in  several  other  taxonomies  into  four 
dimensions,  that  is,  preparation  of  work  accomplishment  (e.g.,  mission  analysis,  planning),  task- 
related  collaborative  behaviours  (e.g.,  coordination,  information  exchange),  work  assessment 
behaviours  (e.g.,  performance  monitoring),  and  team  adjustment  behaviours  (e.g.,  backing  up 
behaviours).  Rousseau  et  al.  also  include  a  sequential  component  into  their  framework,  such  that 
teamwork  dimensions  are  organized  according  to  when  they  will  presumably  achieve  their 
intended  effects  and  contribute  to  team  effectiveness. 

While  most  teamwork  models  and  taxonomies  lack  of  consensus  over  the  factors  that  impact 
teamwork,  all  acknowledge  the  importance  of  the  coordination  and  the  communication  among 
team  members.  These  two  critical  factors  for  teamwork  can  be  described  as  follow: 


DRDC  Valcartier  TR  2010-225 


7 


♦  Coordination:  Coordination  generally  refers  to  the  merging  and  mutual  adjustment 
of  activities  between  team  members  on  one  hand,  and  between  team  members  and 
tools/resources  on  the  other  hand,  in  accordance  with  the  task  to  be  executed  [32]. 
Situating  coordination  in  a  context  of  complex  and  dynamic  situations,  Espinosa, 
Lerch,  and  Kraut  [33]  propose  that  coordination  consists  in  efficiently  managing 
dependencies  between  tasks,  resources  and  team  members.  Cannon-Bowers,  Salas, 
and  Converse  [34]  add  that  shared  mental  models  are  a  key  element  in  explaining 
how  a  team  coordinates,  adapts  and  anticipates,  as  they  provide  a  set  of  knowledge 
and  expectations  about  others’  behaviours. 

♦  Communication:  A  common  definition  of  communication  is  the  exchange  of 
information  between  team  members.  Communication  is  a  key  element  in  team 
functioning  because  it  provides  information,  establishes  interpersonal  relationships 
and  maintains  attention  to  task  monitoring.  When  communication  is  related  to  the 
task  (commands,  orders,  tactical  communication)  it  is  positively  related  to 
performance,  whereas  poorly  performing  teams  seem  to  have  more  difficulty  dealing 
with  uncertainty  (making  more  questions  and  giving  fewer  orders)  and  to  emit  more 
statements  unrelated  to  the  task  (e.g.,  [35;  36;]).  This  is  consistent  with  the  findings 
of  Cannon-Bowers  et  al.  [22]  suggesting  that  extensive  communication  can  be 
disruptive  of  team  performance.  Communication  is  inextricably  linked  with  other 
team  process,  such  as  sharing  knowledge,  team  coordination  and  adaptation.  Due  to 
such  complex  relation,  empirical  studies  of  the  relationship  between  communication 
and  performance  do  not  show  a  consistent  pattern.  MacMillan,  Entin  and  Serafty 
[37]  report  evidence  that  communication  and  coordination  are  highly 
interdependent:  reducing  coordination  requirements  reduces  the  need  for 
communication  and  team  performance  tends  to  increase  in  these  conditions. 


2.2.3  Impact  of  workload  on  teamwork 

Most  research  relating  to  workload  is  based  on  individual  workload  and  very  few  studies  concern 
team  workload.  For  this  reason,  most  of  the  knowledge  concerning  team  workload  comes  from 
research  related  to  individuals.  Bowers  et  al.  [38]  define  team  workload  as  the  relationship 
between  the  finite  performance  capacities  of  a  team  and  the  demands  placed  on  the  team  by  its 
performance  environment.  Workload  can  be  operationalized  in  two  different  ways;  it  can  be 
viewed  either  as  time  pressure  or  as  resource  demand. 

Team  workload  is  more  complex  than  individual  workload  in  that  team  members  must  execute 
two  kinds  of  work  at  the  same  time.  In  addition  to  having  to  carry  out  their  own  specific  tasks 
correctly,  members  must  coordinate  their  tasks  with  other  team  members  and  communicate  with 
them  in  order  to  reach  their  common  objectives.  This  refers  respectively  to  the  concepts  of 
taskwork  and  teamwork  mentioned  earlier. 

Bowers  et  al.  [38]  draw  a  portrait  of  various  variables  which  can  impact  workload  in  a  team 
context.  While  this  list  is  not  exhaustive,  the  authors  sustain  that  these  variables  seem  to  be  the 
most  influential: 

♦  Coordination:  This  variable  is  fundamental  to  both  taskwork  and  teamwork.  There 
is  no  clear  evidence  on  how  coordination  actually  moderates  the  effects  of  workload, 
mainly  because  this  variable  is  difficult  to  manipulate  in  an  experimental  context 
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[38].  Research  seems  to  show  that  higher  coordination  requirements  increase 
operator  workload. 

♦  Communication:  Tasks  which  require  a  lot  of  communication  between  team 
members  produce  an  additional  demand  for  the  operators  [39].  Consequently,  high 
communication  requirements  are  associated  with  high  workload. 

♦  Team  experience  and  maturation:  When  team  members  are  used  to  work  with  the 
same  information  and  on  the  same  tasks,  information  processing  can  change  from 
controlled  processes  to  automatic  processes,  which  require  less  resource  from  the 
operators.  Moreover,  when  an  operator  has  more  experience,  communication  with 
other  team  members  is  more  effective.  Thus,  team  experience  and  maturation 
decrease  workload  [38]. 

♦  Team  training:  Training  seems  to  have  a  high  influence  on  workload.  However, 
there  is  no  direct  evidence  of  this  relation.  Because  training  increases  team 
performance,  it  is  suggested  that  operators  have  more  resources  and  workload  is 
decreased. 

Workload  is  a  critical  variable  in  task  allocation.  In  studies  on  task  allocation,  the  level  of 
workload  usually  determines  conditions  of  the  experiment  (and  not  the  other  way  around)  and 
workload  is  kept  constant  from  the  beginning  to  the  end  of  the  simulation  [40].  Another  key  topic 
in  relation  to  task  allocation  and  workload  is  the  study  of  the  interaction  between  the  level  of 
workload  and  the  organizational  structure  of  the  team.  As  such,  Urban  et  al.  [41]  test  different 
levels  of  workload  (low,  baseline,  high)  on  two  organizational  structures  (hierarchical  and  non- 
hierarchical).  They  find  no  interaction  between  the  level  of  workload  and  the  organizational 
structure  (note,  however  that  they  did  not  consider  workload  distribution  across  individuals).  In 
another  study,  Urban  et  al.  [42]  found  that  a  hierarchical  structure  performed  worse  than  a  non- 
hierarchical  organization,  especially  in  high  workload  conditions. 

2.3  Team  structure 

Although  there  is  good  theoretical  agreement  that  the  structure  of  a  team  greatly  influences 
interdependence  relations  and  team  functioning,  there  is  relatively  little  empirical  work  that 
directly  addresses  the  relationship  between  team  structure  and  team  performance.  There  is  no 
clear  distinction  in  the  literature  between  task  allocation,  role  allocation,  team  structure, 
organization  and  team  architecture.  The  main  idea  behind  all  these  concepts  is  the  distribution  of 
task  demands  among  team  members  in  such  a  way  that  the  organization  is  as  efficient  as  possible. 
Below,  we  discuss  communication  structure,  structural  contingency  theory,  and  focus  on  the 
distinction  between  functional  and  multifunctional  (or  cross-functional)  team  structures  and 
approaches  to  team  design. 

2.3.1  Communication  structure 

Marks,  Zaccaro,  and  Mathieu  [43],  studied  three-member  teams  in  a  Tank  War-Game  simulation. 
Judges  categorised  communications  into  six  classes:  1)  assertiveness,  2)  decision  making  and 
mission  analysis,  3)  adaptability  and  flexibility,  4)  situational  awareness,  5)  leadership,  and  6) 
communication.  The  quality  of  team  communication  was  rated  on  a  scale  ranging  from  1 
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(extremely  poor)  to  7  (extremely  good).  Results  showed  that  high  quality  of  communication  on  all 
dimensions  was  positively  related  to  good  performance  on  the  simulation  task  (see  [23],  for 
similar  results).  In  teams,  communication  and  information  exchange  usually  follow  regular 
patterns.  Figure  3  presents  five  prototypical  patterns: 


Circle 


Chain 


Star  (or  Wheel) 


Web  (or  all  channel  or  Comcom)  'Y' 


Figure  3:  Five  prototypical  communication  patterns. 

These  five  diagrams  represent  the  major  communication  structures  [44].  Each  of  these 
communication  networks  has  implications  for  a  number  of  team  features  -  among  others,  the  style 
of  leadership  adopted  and  the  ability  of  individuals  to  contribute  to  collective  decision-making. 
The  characteristics  of  the  different  communication  networks  are  determined  by  the  extent  of 
‘independence’  and  ‘saturation’.  Independence  refers  to  the  opportunities  for  team  members  to 
take  action  and  to  solve  the  problem  without  relying  on  the  assistance  of  others,  while  saturation 
occurs  when  the  task  places  an  excessive  information  load  (or  other  demands)  upon  a  member  of 
the  network,  which  leads  to  inefficiency.  In  a  centralised  network,  the  central  person  is  more 
likely  to  experience  saturation.  Overall,  the  effectiveness  of  the  team  will  be  influenced  by  the 
appropriateness  of  its  communication  pattern  for  the  type  of  task  to  be  accomplished  [45]. 

Various  characteristics  differentiate  the  communication  structures,  and  the  pattern  of 
communication  adopted  has  significant  implications  for  the  work  of  the  team,  the  degree  and 
quality  of  debate  within  the  team,  and  the  ways  in  which  any  decision  is  taken.  For  instance, 
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communication  networks  can  be  distinguished  based  on  the  centralisation  of  the  information 
exchange.  As  presented  in  Table  1  (below),  whether  a  communication  pattern  is  centralised  or  not 
can  influence  the  information  flow,  the  information  load  as  well  as  the  resulting  performance. 
Generally,  centralised  networks  tend  to  be  superior  on  simple  tasks  whereas  decentralised 
networks  are  superior  on  complex  tasks.  The  presence  or  not  of  a  central  person  in  the  network 
represents  another  factor  that  characterizes  the  communication  structure.  Hence,  in  both  the  Star 
and  Y  patterns,  there  exists  a  focal  person  through  whom  the  essentially  linear  communication 
patterns  pass.  In  patterns  such  as  the  Circle  and  the  Chain,  the  information  flow  is  still  linear  but 
with  no  central  person. 

Table  1  presents  the  characteristics  of  the  information  and  performance  outcome  for  centralised 
and  decentralised  networks  (from  Mullins,  [45]). 

Table  1:  Team  performance  as  a  function  of  communication  network  and  task  complexity. 


Type  of  task  /  network 

Information 

flow 

Information 
load  /  task 
completion 

Performance 

outcome 

Simple  task 

Centralised 

networks  \/P 

(e.g..  Star)  cfti 

To  central  person 

Central  person  can 
perform  task  alone 

Good 

Decentralised  C 

networks 

(e.g.,  Web) 

All  around 
network 

No  one  person  has 
all  the  required 
information 

Poor 

Complex  task 

Centralised  ^ 

networks  '\=f 

(e.g.,  Star)  □  □ 

To  central 

person 

Central  person 
becomes  saturated 

Poor 

Decentralised  □ 

Networks 

(e.g.,  Web)  fr 

All  around 
network 

No  one  person 
becomes  saturated 

Good 

A  contrast  can  be  made  between  open-channel  patterns,  such  as  the  Web  network  pattern,  which 
has  ten  channels,  and  closed  channel  systems  such  as  the  Chain  pattern,  with  only  four  channels. 
In  a  flexible  environment,  there  are  several  factors  that  affect  which  pattern  a  team  may  adopt. 
These  include  whether  or  not  there  is  an  authority  figure  or  nominated  leader  present,  the 
characteristics  of  the  task,  such  as  time  pressure  and  interpersonal  dynamics  of  the  team 
members.  In  face-to  face  teams,  Chains  and  Ys  are  uncommon:  Stars,  Webs  and  Circles, 
generally  a  little  modified  from  the  pure  type,  are  the  usual  patterns.  The  pattern  of 
communication  itself  affects  how  accurately  messages  are  transmitted  and  received,  the  level  of 
task  performance  and  the  amount  of  satisfaction  experienced  by  the  team  members. 
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In  a  military  context,  the  structure  of  communication  is  highly  dependent  on  the  structure  of 
authority.  A  linear  pattern  like  the  Chain  is  more  probable  as  it  corresponds  to  the  usual  chain  of 
command  found  in  most  military  organizations.  This  is  also  the  case  in  a  coalition,  although  the 
complexity  of  the  context  may  imply  more  than  one  type  of  communication  pattern.  Indeed,  as 
there  are  at  least  two  global  levels  of  command  in  a  coalition  (a  more  political  level  and  a  more 
strategic/tactical  level),  the  information  exchange  within  each  level  and  between  the  two  levels 
may  be  based  on  different  communication  structures. 

Leader-centred  teams  such  as  Stars  or  Ys  are  likely  to  perform  specific  tasks  more  rapidly  than 
shared  leadership  teams  such  as  Webs,  but  the  latter  appear  to  cope  more  easily  with  change  and 
to  sustain  higher  levels  of  member  morale.  If  the  task  requires  swift  action  with  a  centre-out 
direction  and  limited  pooling  of  information  and  expertise,  the  Star  will  be  the  most  effective. 
However,  given  a  task  for  which  all  members’  contributions  are  needed,  and  team  morale  needs 
to  be  sustained  over  a  long  period,  the  Web  may  well  work  the  best.  Centrality  (by  circumstance) 
can  help  to  determine  the  leadership  role:  Positions  that  are  lower  than  others  in  communication 
centrality  lack  much  opportunity  for  action  and  influence  and  can  feel  excluded,  and  therefore 
may  be  less  likely  to  take  on  a  leadership  role. 

Baron  and  Greenberg  [46]  as  well  as  Shaw  [47]  suggest  that  the  Wheel  would  produce  an 
effective  result  when  simple  tasks  are  involved.  In  those  cases,  the  central  person  could  undertake 
the  task  alone  with  the  necessary  information  provided  by  the  peripheral  members.  They  also 
propose  that  an  all-channel  pattern  like  the  Web  would  produce  a  poor  result  in  such  situations 
because  the  flow  of  information  circulates  all  around  the  group  with  no  single  person  collating  it 
in  terms  of  decisions  necessary.  The  authors  point  out  that  for  a  complex  task/problem,  these 
outcomes  would  be  reversed.  In  complex  decision  situations,  it  is  usually  necessary  to  have  the 
flow  of  communication  around  the  team  to  encourage  ‘richness’  in  analysis  and  debate  as  well  as 
limiting  the  demands  placed  on  the  leader  to  find  a  solution. 

2. 3. 1.1  Network-centric  organization 

Dekker  [48]  criticizes  the  fact  that  the  network-centric  warfare  literature  advocates  self¬ 
synchronisation  rather  than  centralisation  of  command  and  control  activities  without  considering 
that  there  may  be  specific  circumstances  in  which  one  organizational  structure  is  more  effective 
than  the  other.  Using  an  agent-based  simulation,  Dekker  compared  the  performance  achieved 
using  a  centralized  organizational  structure  in  which  a  command  agent  elaborates  the  plan  for 
solving  the  problem,  or  a  decentralized  organizational  structure  in  which  agents  self-synchronize 
by  exchanging  messages.  This  study  examined  the  efficiency  of  these  two  structures  in  three 
different  problem-solving  tasks.  For  each  task,  the  level  of  time  pressure  and  network  quality  (the 
speed  of  message  exchange)  was  varied  in  systematic  way  in  order  to  ascertain  the  generality  of 
the  simulation’s  outcomes. 

The  simulation  environment  was  populated  by  simple  communicating  agents  providing  an 
abstract  representation  of  communication  and  synchronization  between  military  entities.  The 
agents  faced  three  computational  problems  of  increasing  difficulty: 

♦  The  selection  problem.  One  agent  is  to  be  assigned  to  a  task  and  the  group  must 
select  the  best-qualified  agent.  In  Dekker’ s  abstract  simulation,  each  agent  simply 
has  a  randomly  assigned  suitability  score. 
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♦  The  target  assignment  problem.  This  task  involves  16  or  64  agents  (i.e.,  small 
version  and  large  version)  and  16  targets.  The  most  suitable  agent  must  be  assigned 
for  each  target.  In  this  relatively  simple  version  of  the  task,  the  most  suitable  agent  is 
the  one  closest  to  the  target.  Figure  4  illustrates  this  problem. 


Figure  4:  The  target  assignment  problem  (from  [48]). 

♦  The  “ travelling  general  ”  problem.  This  is  actually  the  travelling  salesman  problem, 
but  with  a  general.  In  this  task  a  General  must  visit  all  agents  in  turn,  and  return  to 
his  starting  point  (see  Figure  5).  The  objective  is  to  find  the  shortest  path  that  may  be 
traveled  in  order  to  “minimize  the  risk”  to  the  general. 


Figure  5:  The  travelling  general  problem  (from  [48]). 

The  simulation  results  demonstrated  that  for  the  selection  problem,  the  centralised  and  self- 
synchronised  approaches  work  equally  well,  but  the  self-synchronised  approach  has  the 
advantage  of  being  fault-tolerant  (the  network  is  resilient  to  agents  going  off-line).  For  the  small 
version  of  the  target  assignment  problem,  when  time -pressure  is  low,  the  centralised  approach  can 
provide  a  “knowledge  edge”  by  finding  the  best  possible  solution.  Nonetheless,  the  distributed 
process  can  perform  as  well  as  the  centralised  process  in  many  versions  of  the  target  selection 
task,  and  actually  performs  better  when  there  is  great  time  pressure  and  a  fast  network. 
Furthermore,  the  decentralised  structure  has  the  advantage  of  fault-tolerance:  it  can  continue  to 
operate  even  though  some  agents  get  disconnected  from  the  network.  For  the  travelling  General 
problem,  the  centralised  and  decentralised  structures  had  comparable  results,  except  when  time 
pressure  was  very  high  and  network  communications  were  very  fast.  In  that  case,  the 
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decentralised  structure  found  shorter  travel  paths  than  the  centralised  structure.  Based  on  these 
findings,  Dekker  [48]  concluded  that  a  combination  of  fast  network  and  extreme  time  pressure 
corresponds  to  the  type  of  situation  in  which  Network  Centric  Warfare  is  most  appropriate. 

Finally,  Dekker  investigated  the  effects  of  using  a  hierarchical  structure  in  the  large  version  of  the 
target  assignment  problem.  Hierarchies  constitute  a  compromise  between  centralisation  and  self¬ 
synchronisation  and  are  widely  used  in  both  military  and  bureaucratic  organizations.  The  64 
agents  were  organized  into  16  teams  (i.e.,  a  multiteam  system)  and  5  additional  command  agents 
completed  the  hierarchical  structure  (see  Figure  6).  This  organization  solves  the  target  assignment 
problem  using  a  “divide  and  conquer”  strategy.  The  higher-level  agents  divide  the  area  of 
operations  into  rectangles  containing  only  four  targets  (see  Figure  7).  This  strategy  decomposes 
the  large  target  assignment  problem  to  sixteen  small  target  assignment  problems. 


Figure  6:  Hierarchy  for  the  target  assignment  problem  (from  [48]). 


Figure  7:  “Divide  and  Conquer”  for  the  target  assignment  problem  (from  [48]). 

Results  showed  that  the  hierarchical  organization  generally  performed  worse  than  the  centralised 
and  distributed  structures.  Dekker  explains  that  the  hierarchical  structure  allowed  little 
opportunity  for  coordination  between  the  “leaves”  of  the  tree.  He  concludes  that  performance  in 
hierarchical  organizations  could  be  improved  by  allowing  coordination  between  the  different 
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teams  of  agents  (i.e.,  engaging  in  inter-team  coordination),  using  either  self-synchronisation  or  a 
centralised  coordination  mechanism. 


2.3.2  Structural  contingency  theory 

A  great  deal  of  research  on  organizational  structure  is  based  on  the  structural  contingency  theory. 
This  theory  stipulates  that  there  is  no  single  best  way  to  organize  a  team;  what  is  important  is  the 
optimal  fit  between  the  organization’s  structure  and  its  environment.  Hollenbeck’s  [49]  theory  on 
organizational  structure  identifies  three  fundamental  characteristics  of  team  structure: 
centralisation,  departmentation  and  adaptability: 

♦  Centralisation:  It  relates  to  the  extent  to  which  responsibility  and  authority  are 
distributed  among  members.  A  centralised  team  typically  includes  a  leader  with  the 
responsibility  to  command  other  team  members.  An  advantage  of  centralisation  is 
that  it  ensures  coordination  and  efficiency  due  to  the  strong  element  of  leader 
control.  In  a  decentralised  team,  responsibilities  and  roles  are  distributed  among 
team  members.  An  advantage  of  decentralisation  is  that  it  ensures  quickness  and 
learning  due  to  the  fact  that  there  is  no  hierarchical  structure  that  individual  team 
members  must  go  through  in  order  to  make  decisions. 

♦  Departmentation:  Organizational  structures  can  be  divided  in  two  categories: 
functional  and  divisional.  In  terms  of  task  allocation,  a  functional  structure  refers  to 
an  organization  where  there  are  many  work  units  specialised  in  one  aspect  of  the 
production.  All  units  put  together  generate  a  finished  product.  This  type  of 
organization  creates  specialised  roles  where  the  level  of  interdependence  between 
work  units  is  high.  Divisional  organizations  consist  in  each  unit  possessing  the  skills 
and/or  resources  required  to  solely  generate  the  product.  Such  work  units  are  capable 
of  working  relatively  autonomously. 

♦  Adaptability:  It  refers  to  the  degree  to  which  teams  are  able  to  adapt  their  structure 
to  take  advantage  of  the  characteristics  of  a  certain  type  of  structural  organization 
(e.g.,  centralised/functional  versus  decentralised/divisional),  depending  on  the  task’s 
requirements  and  context.  Adaptable  teams  are  able  to  adjust  their  structure  to  the 
environment  they  are  facing,  whereas  fixed  teams  tend  to  maintain  one  type  of 
structure. 

Hollenbeck  [49]  extended  the  basic  principle  of  structural  contingency  theory.  He  describes  a 
theory  with  two  complementary  types  of  “structural  fit”:  (1)  the  fit  between  the  organization’s 
structure  and  its  environment  (called  the  external  fit),  and  (2)  the  fit  between  the  organization’s 
structure  and  the  team  members’  skills,  traits  and  expertise  (called  the  internal  fit). 

2.3.3  Functional  vs  multifunctional 

Price,  Miller,  Entin  and  Rubineau  [50]  tested  the  impact  of  team  organizational  structure 
(functional  versus  divisional)  on  performance,  using  humanitarian  relief  scenarios.  In  a  functional 
team  structure,  individuals  have  different  complementary  roles:  Team  members  are  specialized 
and  interdependent.  In  a  divisional  team  structure,  individuals  have  multifaceted  roles  and  are 
relatively  autonomous,  enabling  them  able  to  divide  the  operational  space  into  distinct  areas  of 
responsibility.  Divisional  teams  are  a  particular  case  of  a  multifunctional  team  structure  in  which 
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the  area  of  responsibility  is  divided  a  priori  rather  than  shared  or  divided  spontaneously  during  the 
course  of  a  mission.  Results  show  that  functional  structures  provide  a  higher  performance,  require 
less  time  to  initiate  tasks  and  show  more  coordination.  They  suggest  that  a  functional  structure  is 
best  because  of  the  relative  predictability  of  the  humanitarian  task  environment.  They  conclude 
that  if  a  mission  poses  an  uncertain  and  unpredictable  environment,  it  would  be  better  to  employ  a 
divisional  organization  structure  at  first,  but  as  the  mission  becomes  more  known  and  predictable, 
the  organization’s  efficiency  could  be  improved  by  making  a  transition  to  a  more  functional 
organization. 

Diedrich  et  al.  [51]  evaluated  the  efficiency  of  the  organizational  structure  (divisional  vs. 
functional)  in  relation  to  mission  requirements,  using  simulations  implemented  in  the  Distributed 
Dynamic  Decision  making  (DDD)  environment.  Their  findings  reveal  that  functional  teams  tend 
to  have  a  slight  advantage  in  offensive  operations,  while  the  divisional  teams  tend  to  have  a  slight 
advantage  in  defensive  operations.  However,  coordination  and  situational  awareness  were  not 
better  or  worse  across  the  two  types  of  team  structure. 

2.3.3. 1  Asymmetric  adaptability 

Moon  et  al.  [52]  suggest  that  studies  concerning  organizational  structure  are  too  static;  that  is, 
team  members  are  assigned  to  a  functional  or  divisional  structure  at  the  beginning  of  the  task  and 
the  structure  remains  the  same  until  the  end.  However,  organizations  may  need  to  change  their 
structure  online  in  response  to  environmental  changes.  Therefore,  these  authors  aim  to  test  the 
adaptability  aspect  in  Hollenbeck’s  theory  [49].  They  use  a  DDD  simulation  to  study  the 
adaptation  process  when  changing  from  a  functional  to  a  divisional  structure  and  from  a 
divisional  to  a  functional  structure.  Four-member  teams  switch  between  these  two  stages,  and 
performance  is  measured  in  terms  of  number  of  points  accumulated  for  the  two  stages.  The 
authors  then  calculate  the  difference  between  the  two  stages  to  determine  the  level  of  adaptation. 
They  conclude  that  adaptability  is  like  a  one-way  street:  it  is  easier  and  more  natural  to  switch 
from  a  functional  to  a  divisional  structure  than  the  opposite.  The  authors  suggest  that  it  is  more 
natural  because  it  reflects  an  increase  of  task  scope  for  team  members  over  time  and  because 
norms  developed  in  divisional  structures  regarding  communication  are  counter-productive  when 
switching  to  the  other  structure.  It  appears  that  it  is  easier  to  switch  from  a  functional  to  a 
divisional  structure  because  communication  and  coordination  are  key  elements  for  functional 
structure.  Also,  when  the  first  scenario  is  divisional,  there  is  a  high  level  of  independence  and 
performance  is  affected  by  switching  to  a  structure  with  a  lower  level  of  independence. 

2.3.4  Team  design 

Prominent  work  on  team  design  stemmed  from  the  Manning  Affordability  Initiative ,  sponsored  by 
the  Office  of  Naval  Research.  The  project  involved  complementary  experimental  and  modeling 
initiatives.  First,  a  human-in-the-loop  experiment  using  an  air  defense  warfare  scenario  measured 
the  effectiveness  of  eight  8-person  team  working  with  a  standard  watchstation  design  and 
compared  it  to  the  effectiveness  of  six  teams  of  4  individuals  working  with  an  advanced 
watchstation  design.  The  reduced  teams  working  with  the  advanced  watchstations  with  a  human- 
centered  design  performed  either  as  well  or  better  than  the  baseline  teams  [53]. 
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Data  from  this  human-in-the-loop  experiment  were  used  to  calibrate  and  validate  three  human 
performance  models  used  in  synergy  to  model  individual  and  team  performance:  Team  Optimal 
Design  (TOD),  Integrated  Performance  Modeling  Environment  (IPME),  and  GOMS  Language 
Evaluation  and  Analysis  tool  (GLEAN).  Here,  we  will  focus  on  the  TOD  methodology  since  it  is 
the  most  relevant  for  modeling  the  effects  of  team  structure. 

TOD  [54]  is  a  methodology  for  helping  designers  of  man-machine  systems  to  model  team 
performance  and  to  perform  trade-off  analyses  that  systematically  vary  team  size,  the  capabilities 
of  team  members,  their  responsibilities,  their  technologies,  mission  demands,  and  other  factors. 
TOD  is  based  on  a  multi-phase  allocation  model  that  consists  of  three  parts  [55]: 

1.  the  tasks  and  their  interrelationships  (the  “mission”) 

2.  the  resources  needed  to  accomplish  those  tasks 

3.  the  human  operators  who  constitute  the  team 

Figure  8  illustrates  these  three  parts  and  the  multi-phase  allocation. 
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Figure  8:  Team  optimal  design  methodology  ( from  [55]). 

Kieras  and  Santoro  [56]  discuss  the  lessons  learned  from  computational  modeling  attempts 
performed  during  the  Manning  Affordability  Initiative  project.  Here  are  some  remarks  on  the  key 
challenges  of  this  approach,  stating  that  validating  a  model  against  data  may  be  impractical: 

“Research  on  GOMS  (Goals,  Operators,  Methods,  Selection  rules)  and  other 

modeling  methodologies  has  usually  tested  the  validity  of  the  model  by 

comparing  its  predictions  to  empirical  data  collected  with  actual  human  users 

in  the  same  tasks  and  interface.  [...]  But  a  GOMS  model  is  based  on  a  task 

analysis  that  identifies  the  user’s  goals  and  procedures,  and  can  be  wildly 

inaccurate  if  the  task  analysis  is  wrong.  The  usual  way  to  identify  such  an 
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error  would  be  to  compare  the  model’s  predictions  to  empirical  data. 
However,  the  whole  rationale  for  modeling  human  performance  is  to  reduce 
the  amount  of  empirical  data  collection  needed  to  arrive  at  a  usable  design; 
clearly  if  validating  the  model  requires  as  much  data  collection  as  user 
testing,  there  is  little  point  to  doing  modeling.  We  discovered  that  the 
situation  was  actually  more  serious:  in  this  very  complex  task,  it  was  actually 
impractical  to  collect  enough  data  to  credibly  validate  a  model”  [56]. 


The  solution  proposed  by  Kieras  and  Santoro  is  to  forego  empirical  calibration  and  validation  and 
instead  to  have  subject  matter  experts  criticize  the  behavior  of  the  model. 

“A  scientifically-sound  attempt  to  validate  a  model  for  the  task  would  require 

larger  sample  sizes,  multiple  scenarios,  and  more  experimental  control  over 

team  procedures  along  with  the  same  equipment  and  real  test  users.  The  huge 

expense  of  such  an  effort  means  that  it  could  never  be  done”  [56]. 

We  suggest  that  the  problem  of  model  calibration  and  validation  using  experimental  data 
highlighted  above  could  be  resolved  by  developing  simpler  and  more  tractable  computational 
models.  Computational  GOMS  models  and  complex  cognitive  architectures  [56]  as  well  as  the 
TOD  and  the  IPME  methodologies  described  in  Freeman  et  al.  [53]  are  currently  limited  by  their 
great  complexity.  Such  models  attempt  to  characterize  processes  and  subtasks  with  such  detail 
that  estimating  the  numerous  unknown  parameters  appropriately  would  require  an  excessively 
large  number  of  experimental  manipulations  and  measurements.  This  problem  is  similar  to  the 
notion  of  “sample  complexity”  in  machine  learning:  the  more  complex  the  model,  the  more 
examples  (i.e.,  constraints)  it  requires  in  order  to  make  reliable  predictions.  We  suggest  that  these 
computational  methods  should  be  more  focused  in  order  to  model  specific  problems  more 
effectively  and  reliably. 

2.4  Costs  and  benefits  of  teamwork 

The  preceding  sections  of  this  report  clearly  demonstrated  that  teamwork  is  a  multi-facets  concept 
that  is  affected  by  the  environment  in  which  teamwork  occurs.  The  complexity  of  teamwork 
raised  the  problem  of  identifying  universal  teamwork  behaviours  required  for  optimal  team 
performance.  These  sections  also  showed  the  variability  of  the  impact  of  workload  in  function  of 
the  team  structure. 

Different  team  structures  have  some  disadvantages  and  some  advantages  whose  impact  depends 
on  the  task  to  perform  and  the  environmental  conditions.  Consequently,  the  prediction  of  team 
performance  or  the  identification  of  the  optimal  team  structure  depending  on  the  task  to  perform 
or  the  environmental  condition  represents  a  challenge  for  the  scientific  community.  A  principled 
approach  is  needed  to  model  the  effects  of  team  structure  on  team  effectiveness. 
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2,4.1  Proposal 


The  purpose  of  the  present  project  is  to  develop  a  method  for  predicting  the  effectiveness  of 
different  team  structures.  The  method  we  propose  for  estimating  the  costs/benefits  of  different 
team  structures  combines  two  distinct  approaches: 

1.  A  data-driven  approach,  based  on  the  analysis  of  team  performance  in  a  simulated  task 

2.  A  task-driven  approach,  based  on  the  analysis  of  the  C2  task,  teamwork  requirements,  and 
tool  interactions. 

We  submit  that  modeling  the  cost/benefits  of  teamwork  requires  a  representation  of  the  set  of 
subtasks  associated  to  each  team  member,  of  the  set  of  tools  required  by  each  agent  to  accomplish 
his  role  and  of  the  nature  of  team  interactions.  Manipulating  team  architecture  is  assumed  to 
influence  task  difficulty  in  terms  of  workload  by  altering: 

1.  Each  human  agent’s  specific  taskwork  requirements  (e.g.,  information  that  must  be  gathered, 
units  to  control,  decisions  to  make); 

2.  The  extent  to  which  each  agent  must  interface  with  tools  (e.g.,  necessary  interactions  with 
specific  panels  of  the  interface  or  with  input  devices); 

3.  The  overall  teamwork  requirements  (e.g.,  communication  and  coordination  work  that  arises 
when  interdependent  tasks  are  distributed  across  agents). 

First,  we  will  characterize  the  key  differences  between  functional  and  multifunctional  structures 
in  order  to  explain  how  these  structural  factors  lead  to  differences  in  team  performance.  We  will 
perform  a  task-to-agent  mapping  that  builds  on  the  results  of  the  Hierarchical  Task  Analysis 
(HTA).  This  mapping  will  enable  us  to  identify  interpersonal  task  dependencies  and  the  types  of 
activities  that  must  be  performed  by  each  agent  (taskwork,  teamwork  and  interaction  with  tools). 
The  task-to-agent  mapping  will  provide  the  basis  for  assessing  the  relative  workload  of 
individuals  in  each  team  structure.  Second,  these  structural  factors  will  constitute  inputs  to  a 
mathematical  model  designed  to  predict  team  effectiveness  as  a  function  of  team  structure.  The 
mathematical  model  will  relate  team  structure  to  team  effectiveness  using  a  construct  relevant  to 
all  team  structures:  the  workload  of  each  individual  in  terms  of  teamwork,  taskwork  and  tool 
interaction. 

A  similar  multi-step  methodology  has  been  developed  by  Freeman  et  al.  [53]  in  the  context  of  C2 
in  naval  air  defence  warfare.  However,  we  argue  that  the  computational  GOMS,  TOD  and  IPME 
models  [53;  56],  described  earlier  are  unnecessarily  complex  and  that  this  can  seriously  limit  their 
reliability  when  calibrated  on  limited  experimental  data.  The  time,  effort,  expertise  and  cost 
required  to  develop  and  apply  these  models  may  be  greater  than  necessary.  For  example,  Fevchuk 
et  al.  [55]  model  task  effectiveness  using  many  variables  related  to  individual  characteristics  such 
as  initial  incompetence,  learning  and  memory  (retention  of  learning).  These  free  parameters  are 
not  essential  to  predict  the  effectiveness  of  various  team  structures  and  they  may  provide  too 
much  flexibility  to  these  models,  thus  increasing  the  risk  of  noise  overfitting  [57]. 
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The  present  project  proposes  a  new  framework  specifically  focused  on  estimating  the  costs  and 
benefits  of  teamwork  because  no  prior  framework  is  currently  capable  of  providing  reliable 
quantitative  estimates  of  team  effectiveness  using  a  relatively  simple  and  tractable  approach. 


2.4.2  Methodology 

The  proposed  methodology  for  the  data-driven  part  of  the  analysis  is  to  perform  a  microworld 
experiment  using  a  simulated  C2  task  for  teams  and  manipulating  team  structure  in  order  to 
measure  team  performance  and  various  teamwork  processes  such  as  communication  and 
coordination.  Microworlds  are  task  environments  that  are  used  to  study  behaviour  under 
simulated  conditions  within  a  laboratory  setting  [58].  They  retain  the  basic  or  essential  real  world 
characteristics  while  leaving  out  other  aspects  deemed  superfluous  for  the  purposes  at  hand. 
Microworlds  offer  the  great  advantages  of  experimental  manipulation  and  control,  without 
stripping  away  the  complexity  and  the  dynamic  nature  of  the  task.  As  in  real  world  tasks, 
participants  in  microworld  experiments  have  to  choose  between  different  alternatives  in  order  to 
reach  a  goal  (i.e.,  dynamic  decision  making  and  complex  problem  solving). 

We  opted  for  the  C3Fire  microworld  as  our  testing  platform  [59;  60].  C3  stands  for  command , 
control  and  communications.  C3Fire  is  a  functional  simulation  tailor-designed  to  investigate 
tactical  C2  in  small  teams,  though  the  actual  microworld  simulates  firefighting  rather  than 
military  operations.  C3Fire  seems  best  suited  for  the  functional  simulation  of  tactical  C2 
operations: 

“The  C3Fire  microworld  generates  a  task  environment  in  which  a  group  of  people  cooperate  to 
extinguish  a  forest  fire.  It  can  be  viewed  as  a  command,  control  and  communication  simulation 
environment  that  can  be  used  for  the  investigation  and  training  experimentation  of  team  decision¬ 
making  and  team  Situation  Awareness  (SA).  The  [forest  fire  fighting]  domain  [...]  is  of  subsidiary 
interest  and  has  been  chosen  because  it  creates  a  good  dynamic  environment  [...]  It  is  possible  to 
view  the  generated  session  as  a  simplified  version  of  the  work  tasks  and  the  division  of  labour 
conducted  in  an  emergency  task  or  a  military  task  on  a  tactical  level”  [60]. 

C3Fire  is  appropriate  for  a  functional  simulation  of  military  C2  since  it  involves  time -pressure, 
uncertainty  and  teamwork:  three  key  considerations  for  tactical  C2  teams.  Like  tactical  C2,  the 
simulated  task  requires  dynamic  team  decision  making.  It  involves  regulating  a  dynamic  system 
where:  1)  a  series  of  activities  are  required  to  reach/maintain  the  overall  goal;  2)  activities  depend 
on  the  outcome  of  previous  activities;  3)  task  parameters  are  continuously  varying  in  response  to 
changes;  and  4)  tasks  are  accomplished  in  real  time. 

The  proposed  methodology  for  task-driven  modeling  is  based  on  HTA  [61;  62;  63;  64].  We 
selected  HTA  because  it  is  a  widely  used,  generic  approach  that  forms  the  basis  of  a  number  of 
more  specialized  methods.  HTA  is  readily  understandable  and  provides  few  constraints  on  the 
analysis.  While  the  HTA  will  provide  an  informative  decomposition  of  taskwork,  it  will  not 
identify  activities  related  to  teamwork.  For  that,  a  novel  approach  will  be  presented:  a  task-to- 
agent  mapping  that  will  use  the  results  of  the  task  analysis  to  characterize  the  taskwork, 
teamwork,  interaction  with  tools  and  interpersonal  dependencies  that  follow  from  specific  task 
allocations  and  role  assignments. 
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The  final  part  of  the  methodology  is  to  integrate  results  from  the  data-driven  and  task-modeling 
components  of  the  methodology  to  produce  a  mathematical  model  characterizing  the  relationship 
between  team  structure  and  team  performance.  The  mathematical  model  quantifies  the  impact  of 
each  element  of  the  task-to-person  mapping  on  individual  workload  and  specifies  the  functional 
relationship  between  individual  workload  and  team  performance  while  taking  into  account  task 
dependencies  between  team  members.  Once  calibrated  on  experimental  data,  the  model  can  be 
used  to  predict  the  performance  of  other  possible  team  structures  on  the  same  task. 

2,4.3  C3Fire  microworld 

The  C3Fire  microworld  is  a  simulation  of  a  complex  and  dynamic  task  in  the  firefighting  domain. 
It  is  considered  dynamic  because  the  fire  evolves  autonomously  over  time,  with  or  without  human 
intervention.  The  task  is  considered  complex  because  several  interacting  variables  must  be 
considered  to  make  appropriate  decisions.  The  high  level  of  time  pressure  also  adds  complexity  to 
the  task:  time  taken  to  make  a  decision  or  to  gather  more  information  necessarily  comes  with  a 
cost  since  the  fire  continuously  grows  with  the  passing  of  time. 

The  fire  model  in  the  simulation  is  based  upon  actual  research  on  forest  fires  [58].  This 
microworld  is  designed  to  include  important  characteristics  of  a  real  fire-fighting  system  and  has 
controls  that  allow  researchers  to  adjust  the  complexity  of  the  task  and  manipulate  its  cognitive 
demands. 


Figure  9  :  The  C3Fire  interface  and  operational  environment. 
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The  simulation  takes  place  in  a  40x40  matrix,  representing  a  geographic  map  seen  from  a  bird’s 
eye  view.  Figure  9  shows  the  different  parts  of  the  interface  and  of  the  operational  environment. 
The  red  numbers  mark  distinct  information  panels:  1)  the  role  of  the  player,  2)  game  time,  3)  the 
wind  panel,  4)  pointer  coordinates,  5)  the  unit  information  panel,  6)  the  fire  palette,  and  7)  the 
object  palette.  The  fire  palette  and  the  object  palette  act  as  a  legend  for  the  icons  and  color  on  the 
map.  Units  are  identified  by  a  blue  or  yellow  number  on  the  map.  Blue  units  correspond  to  water 
tankers  and  yellow  units  represent  firefighters. 

The  C3Fire  micro  world  is  built  up  with  a  set  of  four  interacting  simulation  layers:  1)  a  fire  layer, 
2)  a  geographical  objects  layer,  3)  a  weather  layer  and  4)  a  unit  layer. 

Fire  layer.  The  fire  layer  concerns  the  state  of  an  area  which  is  represented  by  a  color  code.  Cells 
may  be  red  (on  fire),  black  (burned  out)  brown  (closed  out)  or  clear.  A  fire  panel  located  on  the 
right  of  the  map  indicates  the  possible  area  states  to  the  participants  (see  Figure  9). 

Geographical  objects  layer.  Four  types  of  geographical  objects  are  differentiated  in  the  C3Fire 
microworld:  houses,  lakes,  pines  and  birches.  Different  objects  may  be  set  to  ignite  faster  or 
slower  than  normal  cells.  An  object  panel,  located  on  the  right  of  the  map,  lists  the  different  types 
of  objects  to  the  participants. 

Weather  layer.  The  weather  layer  is  represented  by  the  force  and  the  direction  of  the  wind.  As  the 
wind  speed  increases,  the  fire  spreads  faster  in  the  direction  the  wind  blows  and  spreads  slower  in 
the  opposite  direction.  The  wind  panel,  located  at  the  upper  right  of  the  map,  indicates  wind 
strength  and  direction  to  the  participants. 

Unit  layer.  The  unit  layer  in  C3Fire  comprises  three  different  types  of  units:  firefighters,  water 
tankers  and  reconnaissance  units.  The  firefighters  combat  fire,  water  tankers  refill  firefighters, 
and  reconnaissance  units  scan  the  map  to  discover  new  fires.  Each  unit  is  represented  by  a  unique 
identification  number.  An  arbitrary  number  of  units  can  be  set  to  appear  in  a  given  C3Fire 
scenario.  Each  unit  awaits  orders  from  its  predetermined  commander.  The  unit  information  panel 
displays  information  on  the  units  controlled  by  a  participant.  This  panel  displays  a  unit’s  identity, 
its  current  position,  destination,  current  activity,  and  current  available  water  cells. 

A  participant  must  left-click  on  a  unit  drag  the  unit’s  “movement  intention  icon”  to  another  area 
in  order  to  order  it  to  go  to  that  destination.  Figure  10  shows  that  when  a  unit  is  moving,  its 
number  or  icon  appears  in  blue  within  the  destination  square. 

Unit's  position  Unit's  intention 

a  V  a 

a  \ : 

N  M 

Unit  moving 


Figure  10:  Moving  units.  Representation  of  position  and  destination. 

The  main  goal  of  the  task  is  to  limit  the  fire’s  progression  by  extinguishing  cells  that  are  on  fire  as 
quickly  as  possible  before  the  fire  spreads  to  adjacent  areas.  To  do  so,  a  firefighter  must  be 
positioned  directly  over  a  burning  cell.  After  a  preset  amount  of  time,  burning  cells  completely 
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burn  out  and  turn  black:  they  are  considered  “lost”.  If  a  firefighter  succeed  in  extinguishing  a  cell 
before  it  burns  out,  then  it  turns  brown  and  is  considered  “saved”.  The  time  required  to  close-out 
a  burning  cell  is  set  by  the  experimenter,  like  the  time  required  to  mobilize  when  arriving  on  a 
new  fire  and  the  time  to  demobilize  once  the  fire  is  out.  A  firefighter  needs  a  certain  amount  of 
water  to  extinguish  a  fire  and  the  water  is  spent  at  a  certain  rate.  To  refill  a  firefighter,  a  water 
tanker  must  be  placed  beside  the  firefighter.  When  a  water  tanker  has  spent  all  his  water,  it  needs 
to  refill  its  water  supply  next  to  a  lake.  The  refill  process  takes  place  at  a  predetermined  speed. 
The  time  needed  for  a  unit  to  move  from  one  cell  to  another  can  also  be  set  by  the  experimenter. 
These  key  parameters  of  the  simulation  can  be  adjusted  to  produce  varying  levels  of  time  pressure 
thus  reducing  or  increasing  task  difficulty. 

Additional  goals  may  be  set  to  make  the  task  more  complex  and  realistic.  For  instance,  protecting 
houses  can  be  considered  a  priority.  Also,  if  participants  do  not  automatically  see  new  fires  on  the 
map  (an  optional  setting),  then  another  important  goal  is  to  use  reconnaissance  units  to  locate 
fires  in  the  operational  environment.  Other  extensions  include  fire-breaker  units  (used  to  block 
fire  progression),  swamp  terrain  (that  cannot  burn),  schools  (high  priority  objects),  and  fuel 
logistics. 

Participants  perform  the  task  on  individual  work  stations.  To  exchange  information,  they  must 
communicate  using  headphones  with  integrated  microphones.  The  key  challenges  in  C3Fire  are  to 
locate  the  fires,  to  select  the  most  important  fires  to  extinguish  first,  to  ensure  that  firefighters  are 
refilled  in  a  timely  manner,  and  to  re-supply  water  tankers  at  appropriate  times.  These  micro¬ 
decisions  form  an  implicit  or  explicit  strategy  for  attaining  both  proximal  and  distal  goals. 

Despite  some  lack  in  the  “face  validity”  of  C3Fire  for  simulating  tactical  C2,  it  is  clear  that  this 
microworld  reproduces  many  of  its  critical  aspects  and  provides  a  challenging  arena  for  team 
decision  making: 

“One  of  the  most  important  advantages  is  that  the  C3Fire  environment  creates 
a  large  motivational  appeal  and  that  user  acceptance  has  been  high  with  the 
professional  subjects  at  the  Swedish  National  Defence  College.  The 
professional  subjects  quickly  adopted  analogies  to  their  own  field  of 
expertise,  and  they  also  often  used  their  professional  “language”  when 
interacting.  This,  and  the  fact  that  many  of  the  teams  clearly  stated  that  they 
found  the  simulation  valuable,  indicates  that  the  microworld,  even  though  the 
simulation  is  fairly  simple,  reflects  some  of  the  crucial  aspects  of  team  work 
in  dynamic  settings”  [60]. 
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Performance  analysis 


This  section  presents  the  data-driven  component  of  the  methodology  for  estimating  the 
costs/benefits  of  teamwork:  data  collection  and  performance  analysis  of  different  team  structures. 
Below,  we  describe  the  objectives  of  the  study,  the  method  of  the  experiment  and  results  of  the 
data  analysis.  We  complete  the  analysis  by  comparing  two  multiple  regression  models  that  show 
key  similarities  and  differences  in  the  functioning  of  the  two  team  structures  that  were  tested. 

3.1  Objectives 

C2  teams  can  be  organized  in  different  ways  to  better  match  the  requirements  of  various  tasks  and 
conditions  [65].  One  challenge  for  team  research  is  to  determine  the  set  of  key  factors  that 
contribute  to  team  performance,  partly  because  these  factors  vary  according  to  the  context  [66]. 
Within  a  same  task  context,  it  is  unclear  whether  factors  of  team  performance  vary  as  a  function 
of  the  organizational  structure  of  the  team.  The  key  objectives  of  the  experiment  reported  here 
are: 

1.  To  measure  the  effectiveness  of  different  team  structures  in  a  dynamic  C2  task 

2.  To  assess  differences  in  team  functioning  as  a  function  of  team  structure. 

The  C3Fire  microworld  provided  the  experimental  testbed  for  studying  C2  in  a  team 
environment.  C3Fire  is  a  dynamic  system,  where  fires  evolve  in  real  time,  both  autonomously  and 
as  a  consequence  of  the  team’s  decisions  and  actions.  The  goal  of  each  team  was  to  extinguish  a 
maximal  number  of  fires  while  attempting  to  save  constructions  in  the  neighbourhood. 

3.1.1  Team  structures 

The  experiment  studied  the  effectiveness  of  three-person  teams  as  a  function  of  team  structure. 
There  were  two  possible  roles  in  the  current  C3Fire  experiment:  firefighting  and  water¬ 
provisioning.  Each  team  member  was  either  specialized  in  one  role  or  performed  both  roles, 
depending  on  the  team  structure.  The  assignment  of  roles/units  to  each  team  member  was  either 
functional  (role-specific)  or  multifunctional  (ranging  over  multiple  team  functions). 

In  the  multifunctional  structure,  each  participant  controls  a  subset  of  firefighters  and  water 
tankers.  Each  participant  is  thus  capable  of  working  relatively  independently.  In  the  functional 
structure  each  team  member  fulfills  a  specific  role.  Team  members  are  highly  interdependent  and 
must  work  together  to  accomplish  the  task.  The  key  difference  between  these  two  structures  lies 
in  their  respective  coordination  requirements.  Table  2  shows  the  specific  allocation  of  units  to 
each  team  member  as  a  function  of  team  structure.  Note  that  the  total  team  resources  (firefighter 
units  and  water  tankers)  are  the  same  for  both  structures.  Any  differences  between  team  structures 
in  terms  of  effectiveness  or  observed  team  processes  (i.e.,  communication,  coordination,  etc.) 
should  therefore  be  attributable  to  the  impact  of  team  structure. 
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Table  2:  Unit  allocation  for  the  three  participants  according  to  team  structure. 


Structure 

Unit  allocation 

STRUCTURE  1 

Agent  X:  Firefighters  =  2,  Water  tankers  =  2 

Agent  Y:  Firefighters  =  2,  Water  tankers  =  2 

Multifunctional 

Agent  Z:  Firefighters  =  2,  Water  tankers  =  2 

STRUCTURE  2 

Agent  X:  Water  tankers  =  6 

Agent  Y:  Firefighters  =  3 

Functional 

Agent  Z:  Firefighters  =  3 

3.1.2  Critical  changes 

Teams  performing  in  command  and  control  environments  are  often  faced  with  sudden  and 
unexpected  events  that  can  modify  the  pace  and  demands  of  the  situation  to  a  great  extent.  To 
function  effectively,  these  teams  must  adapt  to  such  transitions  and  be  efficient  in  coordinating 
their  actions.  The  experiment  was  therefore  designed  to  investigate  how  teams  respond  to  critical 
events  in  C3Fire.  Each  scenario  began  with  a  fire  at  the  map’s  center.  Two  types  of  events  could 
dramatically  influence  the  spread  of  the  fire:  a  change  in  the  strength  and  direction  of  the  wind 
and  the  ignition  of  a  new  fire  elsewhere  on  the  map.  Participants  had  to  detect  these  changes, 
communicate  them  to  their  team  members,  and  adjust  their  behavior  in  consequence. 

3.2  Method 

3.2.1  Participants 

Twenty-four  3 -person  teams  performed  a  2-hour  experiment  including  2  practice  and  4  test 
scenarios  (15  min  each).  Participants  were  recruited  from  the  student  population  at  Universite 
Laval.  They  received  a  15$  compensation  for  participating  in  the  experiment.  Teams  were 
randomly  assigned  to  a  multifunctional  or  functional  organizational  structure.  Figure  11 
summarizes  the  unit  (and  role)  allocation  in  each  team  structure. 
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Figure  11:  Functional  structure  ( left  side):  Each  team  member  controls 
only  one  type  of  unit.  Roles  are  specialized  and  interdependent.  Multifunctional  structure  (right 
side):  Each  team  member  controls  a  subset  of  each  unit  type ,  allowing  them  to  operate 

independently  in  their  joint  effort. 


Participants  were  instructed  to  manage  multiple  goals  in  C3Fire,  ordered  by  priority: 


3.  To  prevent  houses  from  igniting 


4.  To  limit  the  spread  of  the  fire 
To  extinguish  burning  houses 


3.2.2  Material 

The  experimental  took  place  in  a  University  laboratory  with  four  networked  computers  using  3 
programs:  C3Fire,  Team  Speak  and  Morae.  TeamSpeak  software  allowed  communication 
between  team  members;  they  communicated  via  headsets  by  holding  down  the  zero  key  on  the 
numeric  pad.  Participants  could  give  orders  to  their  units  using  the  mouse.  During  scenarios, 
every  event  that  happened  in  the  microworld  (e.g.,  keystrokes),  as  well  as  continuous  screen 
captures  and  communications  between  participants  were  recorded  using  Morae  software 
(TechSmith). 
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3.2.3  Scenarios 


Seven  scenarios  were  designed  for  the  present  study.  A  10  min  scenario  was  developed  for  a  basic 
familiarization  with  the  interface  and  commands.  Two  10  min  scenarios  served  as  practice  trials 
to  ensure  that  participants  were  proficient  in  their  role(s).  Four  15  min  test  scenarios  were  used  to 
study  team  performance.  Two  critical  changes  occurred  during  the  course  of  each  test  scenario. 
At  each  critical  change,  either  a  new  fire  started  somewhere  on  the  map  or  the  wind  increased  and 
changed  direction. 

Table  3:  Critical  changes  in  C3 Fire  for  each  test  scenario. 


Critical  change  1 
(2:19) 

Critical  change  2 
(2:25) 

Test  1 

Fire 

Fire 

Test  2 

Fire 

Wind 

Test  3 

Wind 

Wind 

Test  4 

Wind 

Fire 

Note.  Test  scenarios  began  at  2:15  pm  and  finished  at  2:30  pm  (game  time) 

Thirty  houses  were  placed  in  each  test  scenario  at  different  positions.  Twenty  of  these  houses 
were  positioned  so  as  to  burn  out  before  the  end  of  the  scenario  either  due  to  the  starting  fire  or 
due  to  the  effects  of  a  critical  change.  This  ensured  that  it  was  indeed  critical  to  take  heed  of  these 
changes.  Figure  12  illustrates  a  test  scenario  after  two  critical  changes  (two  new  fires). 


Figure  12:  Example  C3Fire  scenario  with  two  new  fires. 


3.2.4  Procedure 

Participants  were  randomly  assigned  to  a  computer  station  (X,  Y  or  Z)  at  their  arrival  in  the 
research  laboratory.  They  listened  to  instructions  and  followed  directives  from  the  experimenter 
during  a  familiarization  session  aimed  at  explaining  the  objectives  and  how  to  perform  different 
actions  in  C3Fire.  Two  practice  scenarios  followed.  The  order  of  the  four  test  scenarios  was 
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counterbalanced  across  teams.  At  the  end  of  the  experiment  participants  completed  a 
questionnaire  asking  them  to  rate  their  perceived  workload  on  to  dimensions:  mental  load  and 
time-pressure.  These  two  dimensions  of  workload  are  part  of  the  NASA  TLX  workload  rating 
scale  [67].  Each  dimension  was  rated  on  a  scale  from  1  (very  low)  to  10  (very  high). 

3.2.5  Measures 

The  independent  variable  which  was  manipulated  in  the  present  experiment  is  team  structure.  The 
effect  of  team  structure  was  assessed  on  five  dependent  variables: 

5.  Performance  (combination  of  houses  saved  and  closed  out  cells) 

6.  Coordination  (total  time  firefighters  spent  without  water) 

7.  Communication  frequency  and  classification 

8.  Monitoring  effectiveness  (latency  to  detect  critical  changes) 

9.  Perceived  workload  (mental  load  &  time  pressure) 

Performance  in  C3Fire  is  defined  by  the  team’s  success  in  managing  both  the  defensive  and  the 
offensive  aspects  of  the  task,  namely  protecting  the  houses  from  the  fire  and  closing  out  at  many 
fire  cells  as  possible.  Performance  is  calculated  by  multiplying  the  proportion  of  saved  houses  by 
the  number  of  closed  out  cells.  Higher  values  correspond  to  a  better  overall  performance.  The 
proportion  of  saved  houses  corresponds  to  20  (the  number  of  houses  that  are  set  to  burn  out  if 
participants  do  nothing)  minus  the  number  of  houses  which  are  burned  out  at  the  end  of  a 
scenario,  divided  by  20: 

Performance  -  number  of  closed  out  cells  X  ( number  of  houses  saved/20 ) 


Coordination  is  measured  by  the  total  time  each  firefighter  (FF)  spent  without  water.  Lower 
values  correspond  to  a  better  coordination.  This  measure  specifically  refers  to  resource-oriented 
coordination.  This  type  of  coordination  refers  to  process  that  serve  primarily  to  manage 
dependencies  between  activities  or  resources  dependencies  [68].  It  provides  an  excellent 
indication  of  the  efficiency  in  performing  the  water  refill  process,  which  requires  coordination 
between  water  tankers  (WT)  and  FF.  A  second  important  type  of  coordination  not  measured  in 
the  present  experiment  is  goal-oriented  coordination.  Goal-oriented  coordination  is  when 
participants  make  decisions  about  priorities  or  develop  strategies  on  how  to  fight  the  fire  or 
protect  houses. 

Communication  frequency  refers  to  the  total  number  of  messages  sent  by  each  team  member  to 
his  teammates.  These  communications  were  first  categorized  into  seven  communication  topics: 
Fire,  houses,  wind,  actions,  units,  positions,  and  water.  These  topics  were  then  regrouped  into 
those  related  to  resource-oriented  coordination  (units,  positions  and  water)  and  those  related  to 
goal-oriented  coordination  (fire,  houses,  wind,  actions).  This  allowed  a  comparison  of  the 
proportion  of  communications  related  to  each  type  of  coordination. 
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Monitoring  effectiveness  is  defined  as  the  latency  to  detect  critical  changes.  Participants  are 
instructed  to  communicate  to  the  others  the  fact  that  they  have  detected  a  new  fire  or  a  change  in 
wind  strength  and  direction.  Monitoring  effectiveness  is  obtained  by  calculating  the  time  taken 
for  one  of  the  participants  to  notice  the  critical  change  and  communicate  this  observation. 

Perceived  workload  is  measured  only  once  at  the  end  of  the  experiment.  Participants  are  required 
to  rate  the  overall  mental  load  and  time  pressure  that  they  personally  experienced  during  the  four 
test  sessions.  This  measure  can  give  both  an  average  workload  rating  for  each  team,  or  allow  a 
more  detailed  analysis  of  workload  distribution  according  to  the  role(s)  performed  by  different 
participants  (firefighting,  water  provisioning,  or  both). 

3.3  Results 

For  the  following  analyses,  the  measures  of  performance,  coordination,  communication  and 
monitoring  effectiveness  each  team,  we  first  calculated  the  average  results  across  the  four  test 
scenarios. 

3.3,1  Performance 

Team  performance  (closed  out  cells  X  proportion  of  houses  saved)  differed  significantly  between 
the  multifunctional  and  functional  structure,  t( 22)  =  2.92,  p  <  .01.  Multifunctional  teams  obtained 
on  average  a  better  performance  score  (M  =  49.36,  SD  =  12)  than  functional  teams  (M  =  32.73, 
SD  =  15.64).  Table  4  presents  the  results  of  three  comparisons  done  three  distinct  measures  for 
functional  and  multifunctional  teams.  It  decomposes  the  two  element  of  the  performance  measure 
(the  defensive  and  offensive  aspects  of  the  task)  and  shows  that  multifunctional  team  were 
significantly  more  effective  at  saving  houses  (M  =  15.44,  SD  =  1.95)  than  functional  team  (M  = 
11.79,  SD  =  3.98).  They  were  also  significantly  more  effective  at  closing  out  as  many  fires  as 
possible  (M  =  64.5,  SD  =  15.04)  than  functional  ones  (M  =  52.31,  SD  =  11.46). 

Table  4:  Results  of  the  t-tests  comparing  the  two  team  structure  when  decomposing  the  two 

elements  of  team  performance. 


Measure 

T 

Sig.  (2-tailed) 

Houses  saved  (defensive)** 

2,849 

0,009 

Closed  out  cells  (offensive)* 

2,233 

0,036 

Performance  (global)** 

2,921 

0,008 

*  significant  difference  at  p  <  .05 
**  significant  difference  at  p  <  .01 


3.3.2  Resource-oriented  coordination 

The  total  time  FF  units  spent  without  water  differed  significantly  as  a  function  of  team  structure, 
t( 22)  =  -3.485,  p  <  .01.  Time  without  water  was  higher  for  functional  teams  (M  =  2374.17  sec,  SD 
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=  598.81)  compared  to  multifunctional  teams  (Af  =  1649.42  sec,  SD  =  393.49).  Multifunctional 
teams  were  therefore  better  at  coordinating  FF  and  WT  to  satisfy  resource  (i.e.,  water) 
dependencies.  We  found  a  significant  negative  correlation  between  coordination  effectiveness 
(lower  being  better)  and  team  performance,  r(22)  =  -0,59,  p  <  .01. 

3.3.3  Communication  frequency  and  classification 

The  number  of  communications  was  significantly  greater  in  functional  teams  (M  =121,  SD  =  37) 
comparer  to  multifunctional  teams  (M  =  91,  SD  =  29.11),  till)  =  -2.219,  p  <  .05.  The  topics  of 
these  communications  also  differed  in  their  proportions.  Communications  associated  with 
resource-oriented  coordination  (units,  positions  and  water)  occurred  proportionally  more  often  in 
the  functional  teams  than  in  multifunctional  teams,  t(ll)  =  -6.678,  p  <  .01.  Communications 
associated  with  goal-oriented  coordination  (fire,  houses,  wind,  actions)  on  the  other  hand, 
occurred  proportionally  more  often  in  the  multifunctional  teams  compared  to  functional  teams 
t(ll)  =  6.678,  p  <  .01.  Since  these  two  communication  categories  are  mutually  exclusive,  the  t- 
values  of  these  two  comparisons  are  the  same  (but  with  a  different  sign).  Figure  13  shows  the 
proportions  of  these  two  types  of  communication  as  a  function  of  team  structure. 
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Figure  13:  Proportion  of  each  type  of  communication  as  a  function  of  team  structure. 

3.3.4  Monitoring  effectiveness 

The  average  latency  for  detecting  critical  events,  i.e,  the  time  required  for  one  of  the  team 
members  to  detect  and  communicate  the  occurrence  of  a  critical  change,  did  not  differ 
significantly  between  the  two  team  structures,  till)  =  -.199,  ns.  No  difference  related  to  team 
structure  was  found  when  analyzing  detection  latency  for  the  first  critical  change  in  a  scenario, 
till)  =  -1.227,  ns.,  nor  the  second  one,  till)  =  1.034,  ns. 
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3.3.5  Perceived  workload 


No  significant  difference  between  team  structures  was  observed  for  subjective  workload  ratings, 
t(22)  =  -0,72,  ns.  Functional  teams  (M  =  15,13,  SD  =  2,28)  showed  comparable  average  workload 
rating  to  those  of  the  multifunctional  teams  (M  =  14,50,  SD  =  2,04).  However,  workload 
distribution  among  the  team  members  was  not  the  same  in  the  two  organizational  structures.  We 
compared  the  perceived  workload  of  Participant  XF  (the  one  in  the  functional  structure  with  6  WT 
units)  to  the  averaged  perceived  workload  of  Participant  YF  and  ZF,  who  each  controlled  3  FF 
units.  We  performed  the  same  comparison  for  Participant  XM  (in  the  multifunctional  structure) 
and  the  average  workload  rating  of  Participants  YM  and  ZM. 

In  the  multifunctional  structure,  no  difference  was  expected  since  all  three  participants  control  2 
FF  and  2  WT  units.  Indeed,  no  significant  difference  in  workload  distribution  was  found  in  the 
multifunctional  structure,  t( 22)  =  .218,  ns.  However,  for  the  functional  structure,  the  workload 
rating  of  Participant  XF  was  found  to  be  significantly  greater  than  that  of  his  two  teammates,  t( 22) 
=  3.296,  p  <  .01.  Workload  was  distributed  unequally  in  the  functional  structure.  The  reason  for 
not  observing  a  global  difference  in  the  workload  of  the  functional  team  structure  compared  to  the 
multifunctional  structure  is  explained  by  the  fact  that  Participants  YF  and  ZF  (who  control  3  FF 
units)  actually  have  on  average  lower  workload  ratings  that  participants  in  the  multifunctional 
structure  (who  control  a  total  of  4  units). 

A  statistically  signification  correlation  was  found  between  communication  frequency  and  each  of 
the  two  dimensions  of  workload  that  were  measured.  The  higher  the  frequency  of 
communications,  the  higher  the  mental  load,  r(22)  =  .58,  p  <  .01  and  time  pressure,  r(22)  =  .481, 
p  <  .05. 

3.4  Discussion 

The  experimental  results  show  that  team  structure  had  an  important  impact  on  team  performance. 
A  greater  team  performance  was  observed  in  the  multifunctional  structure.  Multifunctional  teams 
were  better  at  resource-oriented  coordination  and  communicated  less  frequently.  These  results  are 
in  line  with  key  finding  from  Stammers  and  Hallam  [69]  suggesting  that  team  structures  need  to 
minimize  the  need  for  interpersonal  coordination  in  order  to  be  more  efficient.  However,  another 
key  finding  in  the  literature  seems  to  contradict  the  above  results.  Functional  teams  tend  to  be 
more  effective  than  multifunctional  teams  in  situations  that  are  routine  and  predictable  [50]. 

Multifunctional  teams  have  the  potential  benefits  of  minimizing  the  interdependence  between 
team  members  and  requiring  little  communication,  but  they  require  managing  different  types  of 
functions  and  therefore  may  require  more  information  and  involve  task  switching  costs.  In  a 
functional  structure,  team  members  may  benefit  from  task  specialization:  the  information 
requirements  are  reduced  and  there  is  a  reduced  task  switching  cost.  However,  interpersonal 
coordination  is  considerably  more  demanding  in  situations  that  are  more  dynamic  and  less 
predictable  and  functional  teams  are  penalized  under  these  conditions  (as  observed  here  and  in 
[70]). 

Our  analysis  suggests  that  there  is  an  apparent  trade-off  between  the  costs  of  interpersonal 
coordination  and  the  benefits  of  task  specialization  in  functional  teams.  This  trade-off  can  be 
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either  beneficial  or  detrimental  to  functional  teams  depending  on  whether  the  task  is  repetitive 
and  predictable  or  not  [71].  Functional  teams  are  more  effective  than  multifunctional  teams  in 
predictable  situations  because  interpersonal  coordination  can  be  achieved  simply  by  relying  on 
routine  procedures  (i.e.,  planned  coordination  that  requires  little  communication).  However, 
interpersonal  coordination  is  considerably  more  demanding  in  situations  that  involve  constant 
change  and  are  less  predictable.  Functional  teams  are  therefore  penalized  under  these  conditions 
because  they  require  more  interpersonal  coordination  than  multifunctional  teams.  In  light  of  the 
trade-off  discussed  above,  it  is  clear  that  C2  teams  should  therefore  adopt  a  structure  that  matches 
the  requirements  of  their  actual  task  and  operating  conditions. 

Two  additional  factors  may  further  explain  the  relatively  lower  performance  of  functional  teams 
observed  in  our  experiment:  workload  imbalance  and  task  dependency.  First,  while  there  was  no 
significant  difference  in  the  overall  workload  of  functional  and  multifunctional  teams,  there  was  a 
workload  imbalance  among  members  of  functional  teams.  Participant  XF,  in  charge  of  controlling 
all  six  water-tankers,  generally  reported  having  a  higher  workload  than  Participants  YF  and  ZF, 
who  were  each  in  charge  of  3  firefighters.  This  result  can  be  easily  explained  by  the  fact  that  a 
distribution  of  4/4/4  units  in  the  multifunctional  structure  may  produce  a  more  balanced  workload 
than  a  distribution  of  6/3/3  units.  It  is  therefore  possible  that  functional  teams  were  disadvantaged 
compared  to  the  multifunctional  teams  in  part  because  Participant  XF  was  overloaded.  Second, 
functional  teams  are  characterized  by  interpersonal  task  dependency .  A  functional  task  allocation 
tends  to  make  some  team  members  depend  on  others  due  to  task  dependency.  In  the  case  of 
C3Fire,  Participants  YF  and  ZF  need  water  from  Participant  XF  to  perform  their  task  correctly,  so 
overloading  Participant  XFcan  be  very  detriment  to  overall  team  performance. 

One  challenge  for  team  research  is  to  determine  the  set  of  key  factors  that  contribute  to  team 
performance  [12].  However,  can  we  assume  that  these  factors  are  constant  for  different  team 
structures?  In  the  present  study,  did  factors  of  team  performance  vary  as  a  function  of  the 
structure  of  the  team?  As  a  final  analysis,  we  addressed  this  issue  by  deriving  two  multiple 
regression  models  from  the  data,  one  for  each  team  structure.  The  predictors  of  performance 
included  in  the  analysis  were  coordination  efficiency,  communication  frequency,  monitoring 
effectiveness  and  two  subjective  workload  measures  (time  pressure  and  mental  load). 

We  found  that  monitoring  effectiveness  was  not  a  useful  predictor  of  team  performance  and  we 
removed  this  factor  from  the  regression  models.  This  variable  provided  no  gain  in  explained 
variance  in  both  regression  models.  We  assessed  the  importance  of  the  other  factors  selected  for 
the  regression  models  by  computing  the  decrease  in  proportion  of  variance  explained  by  the 
regression  models  when  that  variable  is  excluded.  Table  5  shows  the  results  of  this  test  showing 
that  each  variable  is  worth  keeping. 
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Table  5:  Relative  importance  of  different  factors  for  the  two  regression  models  defined  by  loss  of 

explained  variance  when  excluding  that  factor. 


Team  structure 

Predictor  variable  eliminated 

Multifunctional 

Functional 

Resource-oriented  coordination  (time  without  water) 

-0.28 

-0.27 

Communication  frequency 

-0.22 

-0.31 

Mental  load 

-0.07 

-0.11 

Time  pressure 

-0.09 

-0.00 

Note.The  higher  the  reduction  in  proportion  of  variance  explained,  the  greater  the  importance  of  that  factor. 


Both  models  provided  excellent  fits  to  the  data.  The  regression  model  for  multifunctional  teams 
accounted  for  84%  of  the  variance  in  team  performance.  The  second  regression  model  accounted 
for  83%  of  the  variance  in  the  performance  of  functional  teams.  Figure  14  presents  the  scatter 
plots  for  observed  and  predicted  performance  for  each  regression  model. 


Functional  teams  Multifunctional  teams 


Figure  14:  Scatterplots  for  observed  and  predicted  performance  for  each  regression  model. 
Table  6  shows  the  standardized  coefficients  for  each  regression  model. 
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Table  6:  Standardized  regression  coefficients  for  the  multifunctional  and  functional  team 

structure. 


Team  structure 

Predictor  variables 

Multifunctional 

Functional 

Resource-oriented  coordination  (time  without  water) 

-0.82 

-0.61 

Communication  frequency 

-0.75 

0.69 

Mental  load 

0.35 

-0.66 

Time  pressure 

0.37 

0.13 

The  regression  coefficients  showed  similarities  and  differences  in  the  predictors  of  team 
performance  across  team  structures.  There  are  two  main  results.  First,  coordination  was  important 
in  both  team  structures  for  predicting  performance.  Both  models  associate  a  better  (lower) 
resource-oriented  coordination  to  an  increase  in  performance.  Second,  communication  frequency 
was  positively  related  to  performance  in  the  functional  structure  whilst  negatively  related  to 
performance  in  the  multifunctional  structure.  This  result  suggests  that  communication 
requirements  change  as  a  function  of  team  structure.  Indeed,  in  functional  teams,  more 
communications  came  with  a  better  performance  (communicating  was  essential  in  this  structure), 
whereas  in  multifunctional  teams  more  communications  came  with  a  lower  performance.  The 
regression  coefficients  for  mental  load  and  time  pressure  are  somewhat  surprising.  It  was 
expected  that  higher  perceived  workload  would  lead  to  a  reduced  performance.  This  was  only  the 
case  for  mental  load  in  the  functional  team  structure.  We  suggest  that  the  relation  between 
workload  and  performance  may  be  non-linear  and  have  real  detrimental  effects  only  when 
participants  are  overloaded.  This  issue  will  require  further  investigation. 

The  above  findings  suggest  that  teamwork  requirements  and  the  importance  of  various  team 
factors  in  general  may  vary  as  a  function  of  team  structure.  This  novel  result  has  important 
implications.  For  instance,  future  research  should  no  longer  attempt  to  identify  a  general  model  of 
team  performance  that  uses  team  processes  as  predictor  variables  because  the  role  of  these 
processes  is  not  the  same  from  one  team  structure  to  another. 
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4  Task  analysis 


The  C3Fire  microworld  has  provided  an  experimental  testbed  for  studying  dynamic  decision 
making  and  teamwork  in  socio-technical  systems.  The  experiment  reported  in  Section  3  collected 
data  that  will  provide  key  constraints  for  estimating  the  costs/benefits  of  team  structures.  When 
attempting  to  model  a  complex  phenomenon,  it  is  the  constraints  imposed  by  observed  facts  and 
theoretical  knowledge  that  give  meaning  to  the  analysis:  “There  are  so  many  possible  models  that 
the  main  problem  is  not  usually  how  to  construct  a  model,  but  how  to  constrain  the  myriad 
possibilities  down  to  something  useful ”  [72].  Since  the  main  difference  between  team  structures  is 
task  allocation,  it  is  essential  to  carry  out  some  form  of  task  modeling  to  identify  the  set  of 
subtasks  that  can  be  allocated  to  different  participants.  The  task  analysis  presented  in  this  section 
thus  provides  an  important  set  of  constraints  for  the  ulterior  development  of  an  analytical  method 
aimed  at  estimating  the  cost/benefits  of  different  team  structures.  This  is  the  task-driven 
component  of  our  methodology. 

4.1  Objectives 

The  main  goal  of  the  current  analysis  is  to  identify  and  document  the  subtasks  that  must  be 
performed  to  successfully  operate  in  the  C3Fire  microworld.  This  analysis  does  not  focus  on  a 
particular  team  structure:  it  represents  the  generic  tasks  that  must  be  performed  regardless  of  the 
actual  assignment  of  units  and  subtasks,  either  to  a  single  agent  or  to  several  team  members. 

HTA  [61;  62;  63;  64]  is  selected  as  the  method  of  task  analysis  because  it  is  a  widely  used, 
generic  approach  that  is  also  the  basis  of  a  number  of  more  specialized  methods.  HTA  has  the 
benefits  of  being  readily  understandable  and  provides  few  constraints  on  the  analysis.  A 
hierarchical  structure  is  a  central  component  of  most  approaches  to  task  analysis  and  task 
modeling. 

“HTA  is  a  core  ergonomics  approach  with  a  pedigree  of  over  30  years 
continuous  use.  [. . .]  HTA  has  endured  as  a  way  of  representing  a  system  sub¬ 
goal  hierarchy  for  extended  analysis.  It  has  been  used  for  a  range  of 
applications,  including  interface  design  and  evaluation,  allocation  of  function, 
job  aid  design,  error  prediction,  and  workload  assessment”  [64], 


In  the  present  endeavour,  HTA  is  used  to  obtain  a  detailed  view  of  the  C3Fire  subtasks  in  order  to 
characterize  the  total  workload  associated  to  this  task  (also  referred  to  as  taskload). 
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4.2 


Method 


4.2.1  HTA 

HTA  is  focused  on  goal  description  and  decomposition.  Subtasks  are  viewed  as  subgoals  required 
to  achieve  a  higher-level  goal.  A  goal  can  be  active  or  latent.  An  active  goal  is  pursued 
intentionally  throughout  task  execution.  A  latent  goal  can  become  active  at  a  specific  moment 
during  task  execution  when  certain  conditions  arise.  HTA  mainly  consists  of  decomposing  a  main 
task  into  a  hierarchically  organized  set  of  subtasks.  Links  between  different  subtasks  can  be 
formalized  as  plans.  The  main  task  can  be  decomposed  in  as  many  levels  as  needed,  depending  on 
the  degree  of  specificity  desired.  One  possible  aim  of  the  analysis  is  “to  identify  actual  or  possible 
sources  of  performance  failure  and  propose  suitable  remedies”  [73].  Here,  we  use  HTA 
specifically  for  task  description.  Our  analysis  does  not  require  formalized  plans. 

Annett  [73]  defines  seven  steps  in  HTA.  The  first  step  is  to  clearly  define  the  goal  of  the  analysis, 
since  it  influences  how  data  gathering  will  be  done  and  also  has  an  impact  on  the  solutions  which 
can  be  applied.  More  importantly,  the  goal  of  the  analysis  determines  the  depth  of  the  analysis, 
that  is  the  moment  when  enough  information  is  available  and  no  further  decomposition  is  needed. 

The  second  step  in  HTA  is  to  arrive  at  a  consensus  between  all  the  persons  concerned  by  the  task 
on  what  the  performance  goal  is.  Therefore,  a  discussion  with  all  the  stakeholders  is  needed  to 
ensure  that  a  same  view  of  the  task  is  shared,  and  to  identify  differences  in  plans  (ways  to  attain 
the  goal)  that  may  occur  between  them.  During  this  step,  it  is  also  important  to  define  the  precise 
criteria  of  goal  achievement. 

The  third  step  consists  of  identifying  which  sources  of  information  will  be  used  to  perform  the 
HTA.  Those  sources  may  vary  according  to  the  use  of  the  analysis.  The  means  that  are  most 
frequently  used  are  interviews  with  experts  which  can  be  general  or  focused,  direct  observation, 
formal  performance  records  like  flight  recorders,  and  experimental  trials  or  simulations.  In  the 
absence  of  any  performance  data,  hypothetical  questions  are  formulated  which  will  help  estimate 
what  is  at  stake. 

The  fourth  step  involves  drawing  the  HTA  diagram  and/or  HTA  table,  using  the  information 
obtained  in  step  3.  Diagrams  tend  to  provide  a  global  perspective  and  are  easily  understood,  while 
tables  can  be  easier  to  follow  when  the  analysis  has  a  very  large  number  of  elements  and  takes 
multiple  pages.  Tasks  are  assigned  numbers  that  identify  their  position  in  the  hierarchy.  The 
number  0  is  always  attributed  top-level  task  and  numerals  are  assigned  to  each  primary  goal.  A 
digit  is  then  added  for  each  new  level  of  decomposition.  Suboperations  must  be  mutually 
exclusive  and  exhaustive  in  order  to  define  the  superordinate  operation  [73].  Figure  15  shows  an 
example  of  an  HTA  diagram  for  making  a  cup  of  tea.  Figure  16  shows  an  example  in  tabular 
format  for  a  chemical  incident. 
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Figure  15:  FIT  A  diagram  for  making  a  cup  of  tea  (from  [74]). 

0.  Deal  with  chemical  incident 

Plan  0:  Wait  until  1  then  do  2  then  3 -If  [hazard]  then  4  then  5  then  exit  - 
Else  exit 

1 .  [Police  control]  receive  notice  from  public  about  incident  // 

2.  [Police  Control]  gather  information  about  incident 
Plan  2:  Do  2.1  at  any  time  if  appropriate 
Do  2.2  then  2.3 
Then  exit 

2.1.  [Hospital]  inform  police  control  of  casualty 
with  respiratory  problems// 

2.2.  [Police  Control]  get  a  Police  Officer  to  search 
scene  of  incident 

Plan  2.2:  D  o  2.1.1  then  2.2.2  then  2.2.3 
Until  [suspects]  or  [hazards]  then  exit 

2.2.1.  [Police  Control] 

Send  Police  Officer  to  scene  of  incident// 

2.2.2.  [Police  Officer]  arrive  at  scene  of  incident// 

2.2.3.  [Police  Officer]  search  scene  of  incident// 

2.3.  [Police  Control]  get  Police  Officer  to  report  nature  of  incident 
Plan  2.3:  If  [suspects]  then 

2.3.1  If[ suspects]  then  2.3.2.  then  2.3.3 
Then  2.3.4.  then  exit  Else  exit 

2.3.1.  [Police  Officer]  identify  possible  hazard// 

2.3.2.  [Police  Officer] capture  suspects// 

2.3.3.  [Police  Officer]  gathe  r  information  from  suspec  ts // 

2.3.4.  [Police  Officer]  inform  police  control  of  nature  of  incident// 

3.  [Police  Control]  make  decision  about  nature  of  incident// 

4.  [Fire  Control]  clean  up  chemical  spillage 
etc... 

5.  etc... 


Figure  16:  Example  ofHTA  in  tabular  form  [64]. 
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The  fifth  step  in  an  HTA  consists  in  cross-checking  the  validity  of  the  analysis.  The  analysis 
should  be  shown  to  and  discussed  with  the  stakeholders.  Cross-checking  the  validity  of  the  results 
can  also  be  accomplished  by  using  sources  such  as  training  manuals  and  interviews  with 
operators.  This  step  helps  identify  missing  subtasks  and  errors.  This  stage  of  HTA  development 
tests  the  internal  validity  of  the  proposed  task  decomposition. 

The  sixth  step  is  to  identify  the  degree  of  precision  needed  in  relation  to  the  purpose  of  the 
analysis.  If  the  purpose  of  the  analysis  is  to  search  for  sources  of  performance  failure,  it  can  be 
decided  to  stop  the  analysis  when  it  is  not  possible  to  find  further  error  sources.  However,  if  its 
purpose  is  to  define  a  specific  task,  the  analysis  can  be  stopped  when  the  level  of  detail  is 
sufficient  to  provide  a  good  view  of  the  task.  In  this  analysis,  the  purpose  was  to  clearly  define 
the  set  of  subtasks  in  the  C3Fire  microworld. 

The  seventh  and  last  step  consists  of  generating  test  hypotheses,  that  is,  to  invent  situations  that 
would  be  likely  to  occur  and  propose  solutions  to  those  situations.  This  stage  is  basically  a  form 
of  external  validation. 

Though  all  these  steps  were  not  required  for  the  purposes  of  the  present  analysis,  they  have  served 
as  a  guide  for  selecting  the  appropriate  steps  for  task  decomposition.  HTA  is  a  simple  and 
effective  method  for  task  decomposition  widely  used  in  North  America  and  the  UK.  The  major 
advantage  of  this  approach  over  other  forms  of  task  analysis  is  that  its  simplicity  makes  it 
accessible  to  non-specialists. 

4.2.2  Procedure 

The  present  analysis  was  accomplished  by  two  undergraduate  students  having  acquired  extensive 
experience  with  the  C3Fire  microworld,  a  doctoral  student  familiar  with  basic  and  applied 
research  methods  and  a  human  factor  specialist  with  expertise  in  cognitive  modeling  and  task 
analysis.  The  HTA  of  the  task  simulated  in  the  C3Fire  microworld  relied  on  three  information 
sources:  1)  existing  documentation,  2)  observation,  and  3)  SME  interviews.  These  are  the  three 
primary  sources  of  information  generally  used  for  performing  an  HTA  [75].  A  unique  feature  of 
the  present  hierarchical  task  analysis  is  that  the  two  student-analysts  that  were  also  trained  as 
subject-matter  experts  (SME).  The  two  university  students  thus  performed  mutual  observation 
and  cross-interviews  in  order  to  identify  the  different  subtasks  by  taking  in  turn  either  the  role  of 
an  HTA  analyst  or  of  an  SME. 

The  students  gained  expertise  about  C3Fire  by  repeatedly  playing  the  game  during  several  weeks 
in  the  midst  of  a  scenario  calibration  phase  prior  to  experimenting.  Before  the  analysis,  the 
students  did  an  in-depth  learning  of  C3Fire’s  functioning  and  of  the  configuration  of  its  various 
parameters.  They  consulted  the  C3Fire  website  in  order  to  become  well-acquainted  with  the 
features  and  mechanics  of  the  microworld.  Additionally,  they  consulted  selected  scientific  papers 
to  acquire  sufficient  knowledge  of  command  and  control,  dynamic  decision  making  and 
hierarchical  task  analysis. 

In  an  initial  stage  of  the  analysis,  two  versions  of  the  task  decomposition  were  independently 
sketched  and  then  put  together  by  the  subject-matter  experts.  The  SME-analysts  decomposed  the 
game’s  actions  by  recalling  them  from  memory  and  consulting  documentation  on  command  and 
control  (to  stay  in  line  with  the  general  purpose  of  C3Fire  as  a  functional  simulation  of  C2).  Next, 
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the  analysts  collaboratively  identified  and  organized  subtasks  into  a  coherent  hierarchy  during  an 
iterative  development  process.  The  first  draft  was  then  computerized  using  Mindjet  MindManager 
Pro  6.  Meetings  with  the  doctoral  student  and  human  factor  specialist  helped  identify  ways  to 
improve  the  precision  and  internal  coherence  of  the  task  decomposition. 

A  second  stage  of  the  task  analysis  was  to  validate  it  using  replays.  Scenarios  were  recorded  using 
Morae  Recorder  software  version  2.0.1.  These  recordings  allowed  a  session  to  be  captured  on 
video  and  replayed  at  will.  For  the  present  analysis,  two  distinct  scenarios  have  been  captured  in 
this  way.  The  first  scenario  was  played  twice  by  the  SME-analysts  in  order  to  allow  them  to 
switch  roles.  Scenario  1  used  a  functional  team  structure  where  firefighters  are  controlled  by  an 
agent  and  water  tankers  are  controlled  by  the  other  agent.  A  functional  team  structure  was 
selected  because  it  requires  more  coordination  work  across  agents,  making  some  processes  more 
easily  observable  through  communications.  During  the  first  session,  one  expert  controlled  the 
firefighters  and  the  other  controlled  water  tankers.  The  roles  were  then  reversed  for  the  second 
session. 

Scenario  2  familiarized  the  analysts  with  the  subtasks  associated  to  reconnaissance  units.  One 
reconnaissance  unit,  two  firefighters  and  two  water  tankers  were  assigned  to  each  agent  in  a 
divisional  team  configuration.  Each  expert  was  instructed  to  operate  within  the  left  or  right  half  of 
the  map  which  was  divided  in  the  middle  by  a  vertical  blue  line.  Replays  of  Scenario  1  provided 
data  on  which  to  validate  the  initial  task  analysis.  The  validation  was  performed  by  a  cross¬ 
observation  of  the  replays  by  the  two  SME-analysts.  The  first  expert  observed  the  second  expert’s 
replay  and  vice-versa.  Scenario  2  helped  incorporate  the  subtasks  associated  with  reconnaissance 
units,  which  had  not  been  included  in  the  original  analysis.  One  scenario  was  sufficient  to 
perform  a  satisfactory  task  decomposition  related  to  commanding  reconnaissance  units.  No 
further  sessions  were  deemed  necessary  since  the  key  features  of  the  reconnaissance  task  had 
been  successfully  identified.  The  updated  version  of  the  analysis  was  then  revised  and  completed 
in  a  final  meeting  with  the  supervising  analysts. 

Prior  to  showing  results  from  the  HTA,  we  describe  an  additional  analysis  performed  to  extend 
the  description  of  the  workload  associated  with  the  task  simulated  in  C3Fire. 

4.2.3  HTA  extension 

Although  HTA  can  provide  a  detailed  assessment  of  the  various  subgoals  associated  with  a  task, 
other  more  integrative  (and  complex)  approaches  can  be  useful  to  describe  the  work  context  more 
fully.  One  of  these  is  the  Groupware  Task  Analysis  (GTA)  framework,  which  aims  to  combine 
approaches  from  both  human-computer  interaction  (HCI)  and  computer-supported  collaborative 
work  (CSCW)  design  [76].  According  to  this  approach,  analyzing  a  complex  system  requires 
characterizing  the  context  across  multiple  dimensions: 

10.  the  users; 

11.  the  tasks; 

12.  the  equipment  (hardware,  software,  and  materials); 

13.  the  social  environment; 
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14.  the  physical  environment. 

GTA  describes  complex  task  environments  by  focusing  on  agents,  work,  and  situation  elements 
[77]: 

15.  Agents  and  roles.  Description  of  the  active  entities  in  the  task  world  including  users  and 
stakeholders,  systems  and  organizations.  The  roles  and  the  organization  of  work  (i.e.  structure 
of  agents  and  roles)  need  to  be  specified  as  well.  Agents  can  be  characterized  on  relevant 
characteristics  such  as  knowledge,  expertise,  psychological  traits,  etc. 

16.  Work.  The  decomposition  of  tasks,  the  goals,  the  events  that  trigger  the  tasks,  and  the 
different  strategies  used  to  perform  them. 

17.  Situation.  Objects  used  in  the  task  world  as  well  as  their  structure.  History  of  past  relevant 
events,  and  the  whole  social  and  physical  work  environment.  Objects  may  be  physical  or 
conceptual  (e.g.,  messages,  gestures,  etc.). 

Figure  17  illustrates  the  GTA  ontology. 


Used by 


Figure  17:  Ontology  of  Groupware  Task  Analysis. 

The  GTA  framework  was  not  chosen  in  the  present  study  as  a  method  for  modeling  both  the  task 
and  the  social  context  because  the  approach  is  currently  not  sufficient  to  model  teamwork 
processes:  “One  of  the  aspects  most  difficult  to  represent  with  GTA  was  the  differences  due  to 
geographical  distribution  and  the  transfer  of  information  between  different  types  of  users”  [77]. 
Nonetheless,  the  GTA  approach  has  helped  us  identify  useful  elements  to  further  characterize  the 
sub  tasks  identified  in  the  HTA: 

♦  Information  requirements  to  accomplish  the  goal 

♦  Tools  needed  to  access  information  and  perform  actions 

♦  Events  that  trigger  the  goal 
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♦  Actions  required  to  accomplish  the  goal 

Furthermore,  we  draw  from  the  GTA  ontology  the  key  notion  of  roles.  The  different  roles 
identified  in  C3Fire  experiment  are  firefighting,  water-provisioning  and  reconnaissance.  How 
these  roles  are  distributed  across  team-members  will  be  central  to  our  characterization  and 
modeling  of  team  structure  in  Section  6. 

4.3  Results 

Figure  18  presents  the  C3Fire  task  decomposition  in  the  form  of  an  HTA  diagram.  Next,  we 
describe  and  discuss  the  key  elements  identified  in  the  hierarchical  task  analysis  and  its  extension 
based  on  elements  from  the  GTA  framework. 
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Figure  18:  Hierarchical  task  analysis  ofC3Fire. 
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The  task  decomposition  starts  with  a  top  level  that  describes  the  purpose  of  the  C3Fire 
microworld,  which  is  to  simulate  command  and  control  in  a  firefighting  domain.  The  general 
objective  in  terms  of  performance  in  C3Fire  consists  of  controlling  the  spread  of  the  fire  and 
saving  houses.  To  do  so,  the  primary  task  of  the  players  is  to  command  their  units  and  monitor 
their  activities,  that  is,  to  perform  command  and  control.  For  the  purposes  of  the  task  analysis, 
this  main  task  is  divided  in  two  subtasks:  situation  assessment  and  resource  management. 

4.3.1  Situation  assessment 

Situation  assessment  refers  to  having  a  global  view  of  what  is  going  on  in  the  microworld  and 
what  may  happen  in  the  future.  This  process  must  be  performed  continuously  during  the 
execution  of  the  task.  Situation  assessment  can  be  divided  into  three  subtasks:  1)  evaluate  the 
current  situation,  2)  anticipate  the  situation’s  progression  and  3)  assess  threat.  Evaluating  the 
current  situation  is  the  primary  subtask  of  situation  assessment.  It  refers  to  forming  a  global 
image  of  the  operational  environment,  by  knowing  where  the  important  elements  are  located  and 
in  which  state  they  are  at  the  present  moment. 

The  evaluation  of  the  current  situation  is  divided  into  many  subtasks  related  to  cells,  units, 
weather  and  environmental  elements.  These  subtasks  are  basically  performed  in  parallel  and  do 
not  follow  a  sequential  order.  The  first  one,  identify  cell  state,  consists  of  determining  which  cells 
are  on  fire,  which  cells  have  been  closed  out  and  which  cells  have  burnt  out.  This  will  be  useful 
later  to  establish  priorities.  The  second  task  is  to  monitor  the  wind.  This  means  being  constantly 
aware  of  the  current  force  and  direction  of  the  wind.  Evaluate  workforce  status  includes  knowing 
the  position  of  the  units  in  the  microworld,  their  water  supply,  their  current  activity,  and,  if  they 
are  moving,  their  current  destination.  The  last  three  tasks  refer  to  the  location  of  the 
environmental  elements  which  are  the  lakes,  the  forests  and  the  houses.  For  lakes,  it  is  relevant  to 
know  the  proximity  to  water  tankers.  For  houses,  it  is  relevant  to  know  the  proximity  to  fire  cells. 
For  forests  it  is  relevant  to  know  the  type  of  trees  (pines  or  birches)  it  contains. 

In  the  subtask  of  anticipating  the  situation’s  progression,  the  current  knowledge  of  the  situation  is 
used  to  extrapolate  what  the  situation  will  be  in  the  future.  More  precisely,  it  is  possible  to 
anticipate  which  areas  will  be  burning  and  which  units  will  need  water.  Furthermore,  it  is  possible 
to  anticipate  future  unit  states  such  as  when  units  will  become  inactive  (i.e.,  finish  demobilizing), 
when  they  will  be  completely  refilled,  or  when  they  will  arrive  at  a  target  destination. 

The  last  part  of  situation  assessment  is  to  assess  threat,  which  means  being  aware  of  the 
imminence  of  fire  reaching  areas  and  weighting  the  threat  according  to  the  area  type.  This  means 
integrating  knowledge  of  how  far  this  area  is  from  the  fire,  how  fast  the  fire  is  spreading  towards 
it,  which  type  of  cells  are  around  the  area  and  if  they  are  combustible  or  not.  Overall  situation 
assessment  thus  gives  a  global  idea  of  what  is  going  on  within  the  microworld  and  provides 
information  essential  to  the  decision  making  process. 

4.3.2  Resource  management 

Resource  management  refers  mainly  to  choosing  what  to  do  with  each  unit  as  the  situation 
evolves  through  time.  It  basically  refers  to  decision  making  and  planning.  Resource  management 
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is  divided  into  three  subtasks,  all  related  to  the  notion  of  planning:  1)  elaborate  plan,  2)  execute 
plan  and  3)  monitor  plan  outcomes. 

Elaborating  the  plan  consists  mainly  in  making  choices  about  what  fires  to  fight,  what  units  to 
send,  which  units  to  refill  first,  etc.  Elaborating  a  plan  is  decomposed  into  three  subtasks:  1)  select 
area  of  priority,  2)  determine  water  refill  priority  and  3)  select  area  to  explore.  Selecting  the  area 
of  priority  consists  in  choosing  which  fire  to  extinguish  first  and  which  unit  to  dispatch.  These 
choices  a  based  on  the  information  acquired  through  situation  assessment.  Potential  firefighters 
must  be  available  (not  firefighting),  they  must  have  sufficient  water  to  extinguish  a  fire,  and  they 
must  be  within  a  certain  range  to  be  able  to  extinguish  the  fire  in  time.  Determining  water  refill 
priorities  means  looking  for  firefighters  that  need  water  and  selecting  an  appropriate  water  tanker 
to  refill  them.  The  water  refill  process  also  involves  deciding  when  a  water  tanker  must  be  re¬ 
supplied  in  water  and  selecting  a  lake  in  the  vicinity.  Once  more,  situational  information  is 
essential  to  know  which  units  need  water,  their  position,  and  the  location  the  nearest  lake.  Finally, 
selecting  an  area  to  explore  is  important  in  situations  where  reconnaissance  units  are  available  in 
the  scenario  and  fires  are  not  automatically  visible  on  the  map.  A  reconnaissance  unit  must  be 
selected  and  a  dispatch  area  must  be  chosen.  The  role  of  this  unit  consists  in  moving  around  the 
map  and  identifying  fires  within  their  limited  visual  range. 

The  execution  of  the  plan  is  tightly  linked  to  the  elaboration  of  the  plan.  It  is  decomposed  into 
five  actions,  which  all  refer  to  actually  sending  units  to  a  chosen  position  using  the  mouse  and 
dragging  the  unit’s  icon  to  the  desired  position.  Once  at  a  given  position,  the  C3Fire  units 
automatically  start  mobilizing  and  fighting  fires  or  begin  the  water  refill  process.  More  precisely, 
firefighters  are  dispatched  to  water  tankers  to  coordinate  refills,  firefighters  are  dispatched  to 
selected  high  priority  areas,  water  tankers  are  sent  to  lakes  to  refill  themselves,  water  tankers  are 
dispatched  to  firefighters  in  order  to  refill  their  water  supply  and  reconnaissance  units  are 
dispatched  to  the  selected  areas  to  explore. 

Monitoring  plan  outcomes  refers  to  evaluating  the  results  of  the  previous  actions.  It  is  closely 
related  to  situation  assessment  but  implies  selectively  checking  the  results  of  past  actions. 
Monitoring  plan  outcomes  is  divided  into  three  subtasks:  check  for  firefighting  outcome,  check 
for  completed  refills  and  check  for  detected  fires.  Check  for  firefighting  outcome  means  looking 
for  closed-out,  burned  out  or  burning  cells  that  were  targeted,  and  modify  the  plan  if  needed  (i.e., 
the  cell  burns  out)  or  when  the  plan  is  completed  (i.e.,  the  fire  is  extinguished).  Check  for 
completed  refills  means  monitoring  water  levels  in  order  to  optimize  its  efficiency  and  avoid  units 
remaining  idle.  Finally,  checking  for  detected  fires  in  the  reconnaissance  units’  visual  field  (and 
checking  that  the  unit  is  still  moving)  is  necessarily  an  ongoing  activity,  linked  to  the  fire 
detection  subtask  in  situation  assessment. 

The  cyclic  (but  not  strictly  sequential)  nature  of  situation  assessment  and  resource  management 
(represented  in  the  HTA  diagram  shown  in  Figure  18)  is  in  line  with  the  classic  representation  of 
the  military  decision  cycle  called  the  OODA  loop  [5].  OODA  stands  for  a  set  of  four  basic 
processes:  Observe  — >  Orient  — >  Decide  — ►  Act.  Interestingly,  Observe  and  Orient  are  clearly 
aligned  with  the  notion  of  situation  assessment,  while  Decide  and  Act  naturally  correspond  to 
resource  management.  Interestingly,  the  hierarchical  task  analysis  successfully  represented  the 
specifics  of  the  C3Fire  task  while  retaining  the  general  characteristics  common  to  all  command 
and  control  tasks. 
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4.3.3  HTA+  analysis 

For  the  present  research  project,  we  sought  to  complement  the  task  analysis  with  a  specification 
of  information  requirements,  interaction  with  tools,  specific  actions  to  perform,  and  events  that 
trigger  each  subtask  at  level  3  of  the  HTA.  This  additional  analysis  provided  a  more 
comprehensive  characterization  of  the  simulated  C2  task.  The  choice  of  these  four  attributes  to 
complement  the  HTA  analysis  was  inspired  by  the  GTA  [76].  This  extension  of  the  hierarchical 
task  analysis  of  C3Fire,  detailed  below,  is  referred  to  as  the  HTA+  analysis.  This  analysis  was 
developed  by  one  the  SME-analysts  and  revised  by  a  postdoctoral  researcher  with  extensive 
knowledge  of  C3Fire. 

[1.1.1]  Identify  cell  state 

Information:  Cell  color 
Tools:  Map 

Actions:  Scan  map  (move  eyes  on  the  map),  perceive  and  discriminate  brown, 
black  and  red  cells,  make  a  link  between  the  color  of  the  cell  and  its  actual  state 
Event:  Beginning  of  a  scenario,  end  of  a  plan,  periodical 

[1.1.2]  Monitor  wind 

Information:  Wind  speed,  wind  direction 
Tools:  Wind  panel 

Actions:  Check  wind  panel,  identify  the  direction  towards  which  the  arrow 
points,  identify  the  numeric  representation  of  the  wind  speed,  deduce  the  actual 
speed  of  the  wind 

Event:  Beginning  of  a  scenario,  end  of  a  plan,  periodical 

[1.1.3]  Evaluate  workforce  status 

Information:  Unit  position,  unit  activity,  unit's  destination,  unit  level  of  water 
Tools:  MAP,  unit  info  panel 

Actions:  scan  map  to  perceive  and  recognize  units'  icons,  determine  relative 
position  of  the  units,  assess  the  current  activity  of  each  unit,  assess  the  level  of 
water  of  each  unit,  identify  units  that  need  water,  scan  map  to  perceive  unit 
destination  (white  number),  evaluate  distance  between  the  units  and  its  target. 
Event:  Beginning  of  a  scenario,  end  of  a  plan,  current  monitoring 
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[1.1.4]  Locate  residential  areas 


Information:  Houses  position 
Tools:  Map 

Actions:  Move  eyes  on  the  map,  perceive  and  recognize  house's  icons 
Event:  Beginning  of  a  scenario,  end  of  a  plan,  periodical 

[1.1.5]  Locate  lakes 

Information:  Position  of  each  lake 
Tools:  Map 

Actions:  Move  eyes  on  the  map,  perceive  and  recognize  lakes'  icons 
Event:  Beginning  of  a  scenario,  when  refilling  WT 

[1.1.6]  Locate  forests 

Information:  Birches  forest  position,  pine  forest  position 
Tools:  Map 

Actions:  Move  eyes  on  the  map,  perceive  and  recognize  birches'  icons,  perceive 
and  recognize  pines'  icons 

Event:  Beginning  of  a  scenario,  end  of  a  plan,  periodical 

[1.2.1]  Evaluate  fire  progression 

Information:  Cell  states  (clear,  burning,  burned-out,  closed-out) 

Tools:  Map 

Actions:  Assess  overall  size  of  fire(s),  areas  of  containment  and  vulnerable  areas 
Event:  Beginning  of  a  scenario,  current  monitoring 

[1.2.2]  Estimate  future  resources  availability 

Information:  Units'  state  previously  assessed 
Tools:  Unit  info  panel 

Actions:  Use  the  information  previously  found  about  units'  state  to  predict  which 
one  will  soon  need  to  be  refilled  and  which  one  will  be  ready  for  action 
Event:  Beginning  of  a  scenario,  current  monitoring 
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[1.3.1]  Determine  distance  from  fire 

Information:  Fire  position,  Houses  position 
Tools:  Map 

Actions:  Scan  map,  get  a  global  idea  of  the  distance  between  fire  and  houses 
Event:  When  making  plan 

[1.3.2]  Estimate  spreading  speed  and  direction 

Information:  Fire  position,  forests  position,  wind  speed  and  direction 
Tools:  Map,  wind  panel 

Actions:  Predict  the  evolution  of  the  fire  by  taking  into  account  the  force  and 
direction  of  the  wind  and  the  forest  type  present  around  the  current  fire.  Take 
into  account  the  presence  of  closed  out  cells  blocking  fire  progression 
Event:  Current  monitoring 

[1.3.3]  Assess  risk  level 

Information:  cell  contents  (forest,  house),  fire  progression 
Tools:  Map,  wind  panel 

Action:  Integrate  information  to  estimate  strategic  importance  of  a  cell  and  risk 
immediacy.  See  if  there  is  a  house  or  forest  in  the  selected  area,  and  consider  cell 
states  and  contents  in  the  adjacent  area 
Event:  When  making  plan 

[2.1.1]  Select  area  priority 

Information:  Risk  level  of  each  cell 
Tools:  Map 

Actions:  Compare  different  areas  based  on  the  threat  previously  assessed, 
choose  area  that  needs  to  be  protect  first 
Event:  When  making  plan 

[2.1.2]  Determine  water  refill  priority 

Information:  Units'  water  level,  FF  proximity  to  WT,  WT  proximity  to  lakes 
Tools:  Map,  Unit  info  panel 

Actions:  Choose  units  that  need  to  be  refilled  first  based  on  water  level.  To  refill 
a  FF,  compare  distance  of  WTs.  Choose  nearest  WT  with  enough  water  to  refill 
FF.  To  refill  a  WT,  compare  distance  of  lakes  to  WT,  choose  nearest  lake 
Event:  Need  of  refill,  end  of  a  refilling  process 
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[2.1.3]  Direct  terrain  reconnaissance 

Information:  Unexplored  area,  time  passed  since  the  last  recognition  in  a 
determine  area,  position  of  the  rec  units 
Tools:  Map,  unit  info  panel 

Actions:  Look  at  the  position  of  the  rec  units,  find  the  closest  unexplored  area,  or 
the  closest  area  that  has  not  been  explored  recently,  choose  an  area 
Event:  End  of  recognition  in  an  area 

[2.2.1]  Dispatch  FF  to  WT  (To  coordinate  refill) 

Information:  WT  position,  FF  position 
Tools:  Map,  mouse 

Actions:  Click  on  the  chosen  FF  icon,  drag  it  beside  the  chosen  WT 
Event:  One  or  two  empty  FF  near  a  WT  that  can  refill  up  to  two  FF  at  a  time 

[2.2.2]  Dispatch  FF  to  high  priority  coordinates 

Information:  High  priority  coordinates,  FF  position 
Tools:  Map,  mouse 

Actions:  Click  on  the  FF  icon,  drag  the  destination  icon  to  the  priority  area 
Event:  Inactive  units,  new  critical  event,  priority  has  been  established 

[2.2.3]  Dispatch  WT  to  lake 

Information:  Lake  position,  WT  position 
Tools:  Map,  mouse 

Actions:  Click  on  WT  with  the  cursor,  drag  the  destination  icon  beside  the  nearest 
lake  previously  identified 

Event:  Empty  WT,  inactive  WT  with  little  amount  of  water,  priorities  have  been 
established 

[2.2.4]  Dispatch  WT  to  FF 

Information:  FF  coordinates,  WT  coordinates 
Tools:  Map,  mouse 

Actions:  Click  on  the  WT  with  the  cursor,  drag  it  beside  the  FF 
Event:  Empty  FF 
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[2.2.5]  Dispatch  rec  units 


Information:  Unexplored  area,  areas  that  have  not  been  explored  for  a  while 
Tools:  Map,  mouse 

Actions:  Click  on  rec  unit  with  cursor,  drag  it  in  the  chosen  area 
Event:  End  of  reconnaissance  in  a  selected  area 

[2.3.1]  Check  for  firefighting  outcome 

Information:  Cells  state,  FF  position 
Tools:  Map 

Actions:  Check  unit  info  panel,  identify  inactive  FF,  check  map  to  identify  color  of 
the  cell  under  the  unit's  icon,  determine  the  success  or  failure  of  the  FF  action 
Event:  Plan  has  been  executed 

[2.3.2]  Check  for  completed  refills 

Information:  Water  level  of  each  unit,  unit's  activity 
Tools:  Unit  info  panel 

Actions:  Check  unit  info  panel,  identify  units  that  just  finished  refilling  process, 
check  their  water  level  to  assess  the  success  or  failure  of  the  refilling  process 
Event:  Plan  has  been  executed 

[2.3.3]  Check  for  detected  fire  in  rec  units  visual  field 

Information:  Cell  state,  rec  units  position  and  movement  destination 
Tools:  Map 

Actions:  Monitor  if  new  red  cells  appear  in  the  rec  unit's  visual  field,  check  if  rec 
unit  has  reached  its  movement  destination 
Event:  Plan  has  been  executed 

4.4  Discussion 

We  performed  a  hierarchical  task  analysis  of  C3Fire  in  order  to  provide  a  detailed 
characterization  of  the  taskload  involved  in  this  simulated  C2  task.  We  also  extended  this  analysis 
by  identifying  the  information  requirements,  the  tools,  the  events  and  the  actions  associated  with 
each  subtask  in  order  to  provide  a  more  exhaustive  assessment  of  workload  requirements.  This 
work  is  a  key  step  in  developing  a  method  for  estimating  the  costs  and  benefits  of  different  team 
structures. 
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One  aspect  that  is  missing  in  the  current  analysis  due  to  an  inherent  limitation  in  HTA  is  the 
explicit  allocation  of  subtasks  to  individual  agents.  The  allocation  of  subtasks  to  different  team 
members  necessarily  creates  dependencies  between  team  members  that  require  teamwork  (e.g., 
communication,  coordination).  These  dependencies  cannot  be  represented  in  an  HTA  or  any  other 
current  form  of  task  analysis.  This  constitutes  one  of  the  key  challenges  that  will  be  undertaken  in 
the  next  step  of  this  research:  to  develop  an  approach  for  representing  agents,  tasks,  tools  and 
teamwork  in  an  integrative  framework.  The  envisioned  approach  represents  teams  as 
sociotechnical  systems  where  interactive  behaviour  depends  on  1)  the  set  of  subtasks  associated 
to  each  team  member,  2)  on  the  set  of  tools  required  by  each  agent  to  accomplish  its  role,  and  3) 
of  the  nature  and  extent  of  team  interactions.  This  analysis  will  integrate  the  task  decomposition, 
the  HTA+  analysis  and  data  from  the  C3Fire  experiment  in  order  to  develop  a  model  relating 
team  structure,  workload  and  team  effectiveness: 

One  reason  for  the  endurance  of  HTA  is  that  it  can  provide  a  comprehensive  model  of  a  sub-goal 
hierarchy  in  a  system.  [...]  The  sub-goal  hierarchy  lends  itself  to  all  manner  of  analyses,  which  is 
the  real  point  of  HTA.  HTA  was  never  meant  to  be  the  end  point  in  the  analyses,  just  the  start  [64, 
P-77). 

4.4.1  Directions  for  future  research 

Team  tasks  such  as  C2  can  be  characterized  as  cognitively  complex  and  embedded  in  a 
sociotechnical  environment  [78].  We  believe  that  psychological  and  social  processes  are  best 
studied  in  the  context  of  the  Task/Tool/Human  triad  [79;  80]  shown  in  Figure  19. 


Figure  19:  The  TRIAD  framework  [79;  80]. 

The  Rousseau  and  Price  model  is  a  simple  network  in  which  three  nodes  are  linked  on  a  one-to- 
one  basis.  These  nodes  are:  the  Task,  the  Tool  and  the  Human.  The  TRIAD  approach  constitutes 
an  important  step  in  acknowledging  the  role  of  these  three  interacting  elements  in  the  study  of 
distributed  cognition,  human-computer  interaction  and  team  modeling.  Likewise,  Gray  and 
Altmann  [81]  suggested  that  the  behaviour  of  a  joint  cognitive  system  arises  from  the  limits, 
mutual  constraints  and  interactions  between  and  among  each  element  of  the  cognition-artefact- 
task  triad. 
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This  approach  suggests  that  further  advances  in  task  analysis  should  come  from  combining  task 
decomposition  with  a  graphical  representation  of  the  allocation  of  subtasks  across  team  members 
and  an  analysis  of  the  interdependence  that  it  creates  between  human  nodes.  Accordingly,  a 
sociotechnical  task  analysis  should  provide  a  way  to  represent  1)  each  human  agent’s  taskwork,  2) 
the  extent  to  which  each  agent  must  interface  with  tools,  and  3)  the  teamwork  requirements  that 
arise  when  interdependent  tasks  or  unique  tools  are  distributed  across  agents. 

Hollnagel  [82]  developed  the  Functional  Resonance  Accident  Model  (FRAM)  to  analyze 
accidents  or  to  perform  risk  analysis  relative  to  complex  dynamic  systems: 

“Complex  systems,  such  as  socio-technical  systems,  are  by  definition 

composed  of  a  number  of  subsystems,  which  in  turn  may  comprise  multiple 

functions.  Although  the  technological  and  human  (individual,  organisational) 

system  components  are  designed  to  function  in  a  reliable  and  predictable 

manner,  performance  is  always  variable  to  a  smaller  or  larger  extent.  If  a 

subsystem  or  a  component  is  considered  by  itself,  this  performance  variability 

can  be  seen  as  a  weak  modulated  signal,  which  normally  is  undetectable,  i.e., 

it  is  within  the  limits  of  tolerance  of  the  system.  In  relation  to  any  subsystem 

or  component,  the  rest  of  the  system  is  the  environment.  This  environment 

consists  of  a  number  of  subsystems,  for  each  of  which  the  performance  also  is 

variable.  Relative  to  the  subsystem  under  consideration,  the  aggregated 

performance  variability  of  this  “environment”  can  be  understood  as  random 

noise,  and  it  is  this  random  noise  that  can  give  rise  to  resonance,  i.e.,  to  a 

performance  variability  that  is  too  high”  [83], 

Figure  20  illustrates  the  loss  of  system  control  when  performance  variability  is  too  high. 
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Figure  20:  The  functional  resonance  accident  model  [82]. 
The  FRAM  analysis  requires  four  steps: 


18.  Identify  and  characterize  essential  system  functions 

19.  Characterize  the  (context  dependent)  potential  for  variability  using  a  checklist 

20.  Define  functional  resonance  based  on  identified  dependencies  among  functions 

21.  Identify  barriers  for  variability  (damping  factors) 

Though  most  aspects  of  the  FRAM  analysis  are  not  directly  relevant  to  the  present  research,  the 
first  step  -  characterizing  essential  system  functions  -  may  point  to  a  possible  formalism  to  model 
the  workload  of  each  agent  based  on  his  role(s)  and  relation  to  other  agents.  In  FRAM,  the 
characterization  of  each  system  function  is  based  on  a  hexagonal  representation  with  six 
connectors.  Multiple  functions  can  be  related  by  these  connectors  to  form  a  complex  network  of 
functions.  Figure  21  illustrates  the  structure  of  a  FRAM  module. 
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Figure  21:  Generic  FRAM  module. 


The  input  (I)  of  a  function  corresponds  to  the  characteristics  of  the  situation.  The  output  (O) 
corresponds  to  the  result  of  the  activity  or  function.  Preconditions  (P)  represent  conditions  that 
must  be  satisfied  to  accomplish  the  function.  Resources  (R)  necessary  for  the  activity  must  be 
identified,  as  well  as  the  temporal  constraints  (T)  to  accomplish  the  function.  The  sixth  element 
corresponds  to  Controls  (C)  that  serve  to  supervise  or  restrict  the  function.  A  complex  system 
may  invole  many  interrelated  functions  with  feedback  or  feed-forward  connections.  The  output  of 
some  functions  may  act  as  preconditions,  inputs,  resources  or  even  as  a  source  of 
control/influence  to  other  functions. 

Woltjier,  Smith  and  Hollnagel  [84]  performed  a  FRAM  analysis  on  a  subset  of  tasks  required  in 
C3Fire.  Figure  22  illustrates  the  resulting  functions  and  their  interrelation. 


52 


DRDC  Valcartier  TR  2010-225 


(5) 


Burning  vegetation 
and  buildings 

Staff  nnembers 
control  fire  trucks’ 
movements 


/  ^  Saved/lost 

vegetation 

.  \  /  Fire  trucks  V  Fr\\  and  buiidinge 

fight  fire  / 


A 

^  Water/ 


Mrs  truck 
collocated 
with  fira 


/  Fire  truck 


/ 


(dynamics^  (V)— <  moves  to  fire  )— (o) 

\  location  / 


.  Fire  truck  has  water 


©  p 

\ 

Fire  truck 


refills  water 


\ 


y 

® 


\ 


\ 


@ 


— Fuel  /' 


® 


Figure  22:  FRAM  analysis  of  C3 Fire. 

The  FRAM  analysis  provides  a  useful  way  to  represent  functions  and  constraints  related  to  task 
execution.  The  interdependence  between  subtasks  in  particularly  well  described.  However,  the 
FRAM  analysis  of  C3Fire  is  not  ideal  to  characterize  teamwork  since  it  is  centered  on  functions 
rather  than  on  individuals.  A  variant  of  this  approach  with  agents  as  the  focal  point  would  be 
more  appropriate  to  characterize  the  workload  of  each  individual  as  a  function  of  his  role(s).  The 
role(s)  of  an  agent  would  determine  the  tasks/functions  that  he  must  accomplish,  the  prerequisites 
for  performing  these  functions,  the  tools  required,  and  the  teamwork  necessary  to  achieve  the 
collective  goal.  This  would  provide  an  excellent  basis  for  estimating  the  workload  of  individuals 
and  for  representing  dependence  relations  between  team  members  as  a  function  of  team  structure. 
Such  an  approach  will  be  developed  in  Section  5. 
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5  Integration 


This  final  section  combines  elements  from  the  statistical  modeling  approach  developed  in  Section 
3  and  the  results  of  the  task  analysis  from  Section  4  in  order  to  develop  a  new  mathematical 
model  on  how  team  structure  impacts  team  performance.  First,  the  HTA  results  are  used  to 
characterize  the  taskwork  of  participants  as  a  function  of  their  role  within  the  team  structure 
(firefighting,  water-provisioning,  or  both).  Second,  the  Information-Tools-Events-Action  analysis 
performed  to  extend  the  HTA  helped  identify  the  tools  that  participants  must  interact  with 
depending  on  their  role.  The  HTA+  analysis  thus  refers  to  the  combination  of  these  two  analyses. 
These  elements  are  integrated  to  form  a  task-to-agent  mapping  inspired  from  the  FRAM  [84]  and 
GTA  [76]  frameworks  and  can  be  considered  an  extended  form  of  the  sociotechnical  TRIAD  [79] 
described  earlier  in  this  report.  The  task-to-agent  mapping  also  identifies  task  dependencies 
between  participants  and  teamwork  requirements  that  result  from  these  dependencies  (i.e., 
resource-oriented  coordination),  together  with  other  teamwork  processes  relevant  for  the 
simulated  C2  task. 

The  mathematical  modeling  approach  adopted  here  consists  in  estimating  the  workload  of  each 
participant  based  on  the  task-to-agent  mapping  and  modeling  the  relation  between  workload  and 
individual  performance.  Next,  the  model  relates  individual  performance  to  team  effectiveness  by 
taking  into  account  interpersonal  dependencies  imposed  by  the  team  structure.  After  calibrating 
the  model  using  results  from  the  C3Fire  experiment,  we  demonstrate  how  it  can  be  used  to 
estimate  the  effectiveness  of  other  possible  team  structures.  We  conclude  with  a  discussion  of  the 
potential  and  limitations  of  the  present  approach  and  propose  directions  for  future  work. 

5.1  Basic  assumptions  of  the  model 

The  experimental  findings  reported  in  Section  3,  together  with  other  results  from  the  team 
literature  [50;  70;  71;  85],  show  that  there  is  an  apparent  trade-off  between  the  costs  of 
interpersonal  coordination  and  the  benefits  of  task  specialization  in  functional  teams.  This  trade¬ 
off  can  be  either  beneficial  or  detrimental  to  functional  teams  depending  on  whether  the  task  is 
repetitive  and  predictable  or  not  [71].  Functional  teams  are  more  effective  than  multifunctional 
teams  in  predictable  situations  because  interpersonal  coordination  can  be  achieved  simply  by 
relying  on  routine  procedures  (i.e.,  planned  coordination  that  requires  little  communication). 
However,  interpersonal  coordination  is  considerably  more  demanding  in  situations  that  involve 
constant  change  and  are  less  predictable.  Functional  teams  are  therefore  penalized  under  these 
conditions  because  they  require  more  interpersonal  coordination  than  multifunctional  teams. 

In  the  present  study,  our  effort  is  focused  on  C2  team  effectiveness  in  complex  and  dynamic 
situations  (see  also  [86]).  The  cost  of  interpersonal  coordination  is  thus  expected  to  be  high  due  to 
the  rapidly  changing  nature  of  the  situation.  Still,  there  are  two  additional  factors  that  may  further 
explain  the  rather  low  performance  of  functional  teams  observed  in  our  experiment:  workload 
imbalance  and  task  dependency.  First,  while  there  is  no  significant  difference  in  the  overall 
workload  of  functional  and  multifunctional  teams,  there  is  a  workload  imbalance  among  members 
of  functional  teams.  Participant  XF,  in  charge  of  controlling  all  six  water-tankers,  generally 
reports  having  a  higher  workload  than  Participants  YF  and  ZF,  who  are  each  in  charge  of  3 
firefighters.  This  result  can  be  easily  explained  by  the  fact  that  a  distribution  of  4/4/4  units  in  the 
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multifunctional  structure  would  produce  a  more  balanced  workload  than  a  distribution  of  6/3/3 
units.  It  is  therefore  possible  that  functional  teams  were  disadvantaged  compared  to  the 
multifunctional  teams  in  part  because  Participant  XF  was  overloaded.  Second,  functional  teams 
are  characterized  by  interpersonal  task  dependency.  A  functional  task  allocation  tends  to  make 
some  team  members  depend  on  others  due  to  task  dependency.  In  the  case  of  C3Fire,  Participants 
Yf  and  ZF  need  water  from  Participant  XF  to  perform  their  task  correctly,  so  overloading 
Participant  XF  can  be  very  detriment  to  overall  team  performance.  Individual  workload  and 
interpersonal  dependency  will  lie  at  the  heart  of  the  mathematical  model  described  later  in  this 
section. 

The  mathematical  model  has  a  different  purpose  than  the  regression  models.  The  two  regression 
models,  one  for  functional  teams  and  one  for  multifunctional  teams,  highlight  key  similarities  and 
differences  in  the  dynamics  of  these  two  team  structures.  The  analysis  related  team  processes  or 
states  (coordination,  communication,  and  perceived  workload)  to  team  performance  and 
suggested  that  communication  requirements  change  as  a  function  of  team  structure.  That  was  a 
novel  result  with  important  implications.  For  example,  future  research  should  no  longer  attempt  to 
identify  a  general  model  of  team  performance  that  uses  team  processes  as  predictor  variables 
because  the  role  of  these  processes  is  not  the  same  from  one  team  structure  to  another. 

Since  the  regression  models  seem  to  be  structure-specific,  they  cannot  constitute  appropriate  tools 
for  predicting  the  effectiveness  of  other  team  structures.  Another  limit  of  the  multiple  regression 
approach  is  that  the  predictors  of  team  performance  require  measurements  of  team  activity.  Such 
an  approach  would  be  of  little  use  for  predicting  the  effectiveness  of  different  team  structures 
since  they  would  have  to  be  tested  in  order  to  obtain  the  value  of  the  regression  predictors 
(coordination,  communication,  etc.).  What  is  needed  here  is  a  set  of  structural  factors  that  can  be 
used  to  determine  in  advance  the  expected  effectiveness  of  a  team  [55;  87;  88]. 

The  mathematical  model  does  not  attempt  to  account  for  the  different  performance  of  each  team 
in  the  experiment  as  a  function  of  team  processes  like  the  regression  models  did.  Instead,  the 
model  seeks  to  predict  the  average  performance  of  teams  as  a  function  of  team  structure.  The 
model  does  not  attempt  to  explain  differences  between  teams  having  a  same  team  structure  (other 
non- structural  factors  are  responsible  for  this,  e.g.,  motivation,  skill,  cohesion,  leadership,  etc.). 
Nonetheless,  an  important  similarity  between  the  earlier  regression  models  and  the  mathematical 
model  proposed  here  is  that  both  use  a  similar  method  to  estimate  unknown  parameters.  Both  use 
a  least-squares  minimization  method  to  reduce  as  much  as  possible  the  difference  between  model 
predictions  and  empirical  data  [89]. 

The  purpose  of  the  mathematical  model  is  to  provide  a  tool  for  predicting  the  effectiveness  of 
different  team  structures.  This  method  for  estimating  the  costs/benefits  of  different  team 
structures  combines  the  two  distinct  approaches  described  in  the  previous  sections:  1)  the  analysis 
of  team  performance  in  a  simulated  task  (i.e.,  C3Fire)  using  statistical  modeling  and  2)  the 
analysis  of  the  C2  task  simulated  in  C3Fire  using  hierarchical  task  decomposition.  First,  we  will 
characterize  the  key  differences  between  functional  and  multifunctional  structures  in  order  to 
explain  how  these  structural  factors  lead  to  differences  in  team  performance.  We  will  perform  a 
task-to-agent  mapping  that  builds  on  the  results  of  the  HTA.  This  mapping  will  enable  us  to 
identify  interpersonal  task  dependencies  and  the  types  of  activities  associated  with  each  agent 
(taskwork,  teamwork  and  interaction  with  tools)  will  provide  the  basis  for  assessing  the  relative 
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workload  of  individuals  in  each  team  structure.  Second,  theses  structural  factors  will  constitute 
inputs  to  the  mathematical  model  described  afterwards. 

The  model  will  attempt  to  account  for  team  effectiveness  by: 

22.  Estimating  individual  workload; 

23.  Mapping  workload  to  individual  effectiveness; 

24.  Constraining  individual  performance  as  a  function  of  dependency  on  others; 

25.  Aggregating  the  constrained  effectiveness  of  each  individual. 

Part  of  the  model  attempts  to  describe  the  relation  between  the  task-to-agent  mapping  and 
perceived  workload.  A  second  part  aims  to  map  workload  (associated  with  taskwork,  teamwork 
and  interaction  with  tools)  to  individual  performance  using  a  nonlinear  transfer  function  described 
later.  A  third  part  of  the  model  relates  individual  effectiveness  to  team  effectiveness  while 
considering  the  influence  of  task  dependency  between  team  members  (e.g.,  a  participant 
controlling  only  firefighters  needs  water  from  another  participant  with  water-tankers). 

The  model  developed  in  this  section  provides  a  theoretical  account  of  the  effects  of  team  structure 
on  team  effectiveness.  Taskwork  is  distributed  in  different  ways  among  team  members  depending 
on  team  structure.  This  leads  to  different  dependency  relations  and  teamwork  requirements  as  a 
function  of  team  structure.  The  mathematical  model  aims  to  relate  team  structure  to  team 
effectiveness  using  a  construct  relevant  to  all  team  structures:  the  workload  of  each  individual. 
The  concept  of  workload  includes  (1)  the  various  subtasks  assigned  to  the  participant,  but  also  (2) 
the  interaction  with  tools  and  (3)  the  teamwork  necessary  to  accomplish  individual  and  team 
goals.  A  basic  assumption  of  the  model  is  that  for  any  team  structure,  if  all  participants  (correctly) 
perform  their  respective  teamwork  and  taskwork,  using  the  tools  required  for  these  actions,  and 
then  team  performance  shall  be  optimal.  We  assume  that  what  reduces  team  performance  is  the 
difficulty  to  accomplish  this  work  when  team  members  suffer  from  high  levels  of  workload. 

In  summary,  the  present  theoretical  perspective  proposes  a  task-to-agent  mapping  based  on  the 
HTA+  analysis  in  order  to  provide  a  qualitative  assessment  of  the  workload  of  each  participant  as 
a  function  of  team  structure.  The  mathematical  model  quantifies  the  impact  of  each  element  of 
that  mapping  on  individual  workload  and  specifies  the  functional  relationship  between  individual 
workload  and  team  performance. 

5.2  Task-to-agent  mapping 

In  order  to  model  the  effects  of  team  structure,  it  is  necessary  to  identify  how  various  structures 
differ.  The  essential  difference  between  team  structures  is  how  sub-tasks  are  allocated.  However, 
different  task  allocations  also  have  an  impact  on  the  teamwork  required  to  accomplish  the  task. 
An  integrative  representation  of  the  workload  of  each  individual  must  therefore  include  both  the 
taskwork  and  teamwork  required  to  accomplish  the  mission  [29;  31;  90].  Task  allocation  also 
influences  the  tools  that  each  participant  must  interact  with  and  the  dependency  relations  between 
participants.  The  task-to-agent  mapping  aims  to  characterize  the  demands  made  on  each 
individual  along  each  of  these  four  dimensions. 
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A  first  step  toward  this  goal  was  to  perform  a  hierarchical  task  analysis  to  decompose  the  C3Fire 
task  into  a  set  of  subtasks.  We  also  complemented  the  task  analysis  with  a  specification  of 
information  requirements,  interaction  with  tools ,  specific  actions  to  perform,  and  events  that 
trigger  each  subtask  (at  level  3  of  the  HTA).  This  additional  analysis  provided  a  more 
comprehensive  characterization  of  the  simulated  C2  task.  The  choice  of  these  four  attributes  to 
complement  the  HTA  analysis  was  inspired  by  the  FRAM  framework  [84]  and  the  Groupware 
Task  Analysis  [76].  The  combination  of  these  two  methods  constitutes  the  HTA+  analysis. 

There  are  two  possible  roles  in  the  current  C3Fire  experiment:  firefighting  and  water¬ 
provisioning.  Each  team  member  can  either  specialize  in  one  role  or  perform  both  roles, 
depending  on  the  team  structure.  Before  performing  the  task-to-agent  mapping,  we  first 
associated  each  subtask  in  the  HTA  to  each  role  based  on  the  types  of  units  that  the  participant 
controls.  We  also  matched  the  tools  previously  identified  in  the  HTA+  analysis  as  necessary  to 
perform  each  subtask  to  each  role.  For  the  present  analysis,  we  focused  on  subtasks  at  level  3  in 
the  HTA  rather  than  level  four.  This  level  was  the  focus  of  the  HTA+  analysis  because  some 
subtasks  were  not  decomposed  beyond  level  3  in  the  HTA.  This  is  the  finest  level  of  detail  that 
could  be  selected  for  the  present  analysis.  Figure  23  presents  the  results  of  this  task-tool-role 
mapping. 


1.1.  Evaluate  current  situation 


1 .  Situation 


Command  and 
control  in 
firefighting 
domain 


1.1.1.  Identify  cell  state  [MAP]  (FF) 

1.1.2.  Monitor  wind  [WIND]  (FF) 

1.1.3.  Evaluate  workforce  status  [UNIT,  MAP]  (FF+WT) 

1.1.4.  Locate  residential  areas  [MAP]  (FF) 

1 .1 .5.  Locate  lakes  [MAP]  (WT) 

assessment  j-  1.1.6.  Locate  forests  [MAPUFF) 

1 .2.1 .  Evaluate  fire  progression  [MAP]  (FF) 

1 .2.  Anticipate  situation  progression  1,2.2.  Estimate  future  resources  [UNIT]  (FF+WT) 

1.3.1.  Determine  distance  from  fire  [MAP]  (FF) 

1.3.2.  Estimate  spreading  speed  and  direction  [MAP,  WIND]  (FF) 

1 .3.3.  Assess  risk  level  [MAP,  WIND](FF) 


1.3.  Assess  threat 


2.  Resources  management 


2.1.1.  Select  area  priority  [MAP]  (FF) 

2.1.  Elaborate  plan  2.1.2.  Determine  water  refill  priority  [UNIT,  MAP]  (WT) 

2.2.1.  Dispatch  FF  to  WT  (to  coordinate  refill)  [MOUSE,  MAP]  (FF) 

2.2.2.  Dispatch  FF  to  high  priority  coordinates  [MOUSE,  MAP]  (FF) 

2.2.  Execute  plan  2.2.3.  Dispatch  WT  to  lake  [MOUSE,  MAP]  (WT) 

[  2.2.4.  Dispatch  WT  to  FF  [MOUSE,  MAP]  (WT) 

2.3.1 .  Check  for  firefighting  outcome  [UNIT,  MAP]  (FF) 

2.3.  Monitor  plan  outcomes  2.3.2.  Check  for  completed  refills  [UNIT]  (FF+WT) 


Figure  23:  Edited  HTA  representation,  with  tasks  associated  to  specific  roles  and  tools. 

FF  =  firefighting  role,  WT  =  water-provisioning  role,  MAP  =  Geospatial  information  display, 
UNIT  =  Unit  information  panel,  WIND  =  Wind  information  panel,  MOUSE  -  Computer  mouse. 

While  the  HTA  provided  an  informative  decomposition  of  taskwork,  it  did  not  identify  activities 
related  to  teamwork.  Four  teamwork  processes  commonly  found  in  the  literature  were  identified 
by  studying  communication  recordings  from  the  C3Fire  experiment:  Information  sharing,  backing 
up  behaviours,  goal-oriented  coordination  and  resource-oriented  coordination.  These  activities 
can  be  defined  as  follow: 
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♦  Information  sharing  [91;  92]  occurs  for  example  when  a  participant  detects  and 
communicates  a  critical  change  such  as  a  new  fire  or  a  change  in  wind  strength  and 
direction. 

♦  Backup  behaviors  [93]  include  directing  a  teammates’  attention  to  an  unattended 
unit,  making  suggestions  on  how  to  perform  better,  finish  extinguishing  another 
teammates’  fire  when  he/she  runs  out  of  water. 

♦  Goal-oriented  coordination  is  when  participants  make  decisions  about  priorities  or 
develop  strategies  on  how  to  fight  the  fire. 

♦  Resource-oriented  coordination  refers  in  C3Fire  to  the  need  to  communicate  the 
need/offer  for  water,  provide  unit  position  and  confirm  one’s  intentions  when  the 
firefighting  and  water-provisioning  roles  are  distributed  in  an  exclusive  manner 
between  team  members. 

Our  distinction  between  these  two  forms  of  coordination  is  inspired  by  Coordination  Theory  [68]: 

“According  to  coordination  theory,  [team  activities]  can  be  separated  into  those 
that  are  necessary  to  achieve  the  goal  of  the  process  (e.g.,  that  directly  contribute 
to  the  output  of  the  process)  and  those  that  serve  primarily  to  manage  various 
dependencies  between  activities  and  resources”  (p.159). 

Three  important  tools  in  C3Fire  were  not  identified  in  Figure  23  because  they  are  related  to 
teamwork  rather  than  taskwork.  The  coordinate  system  (letters  and  numbers  on  the  left  and  top  of 
the  geospatial  map)  and  the  pointer  position  panel  are  essential  when  two  individuals  must 
coordinate  the  movements  of  their  FF/WT  units  to  perform  a  water  refill  (team  members  do  not 
see  their  respective  units  on  the  map  unless  they  are  very  close).  While  the  participant  controlling 
a  FF  unit  can  simply  provide  the  FF  unit’s  coordinates  to  his  teammate  using  the  information 
displayed  in  the  unit  info  panel,  the  one  sending  the  WT  unit  to  that  position  must  use  the 
coordinate  system  and  the  pointer  position  panel  to  dispatch  his  unit  to  the  correct  location.  The 
third  tool  is  the  communication  button  (together  with  the  headphones),  which  is  essential  for  goal- 
oriented  and  resource-oriented  coordination,  information  sharing  and  backing  up  behaviours. 

Figures  24  and  25  show  the  task-to-agent  mapping  of  the  multifunctional  and  the  functional  team 
structures  along  the  four  dimensions  hypothesized  as  key  structural  factors  of  team  workload  and 
effectiveness:  taskwork,  teamwork,  tool  interaction,  and  prerequisites  (i.e.,  interpersonal  task 
dependency).  Taskwork  is  subdivided  in  two  distinct  functions:  situation  assessment  and  resource 
management.  Information  is  acquired  through  situation  assessment.  Participants  who  perform 
many  roles  (i.e.,  who  control  both  FF  and  WT)  have  more  information  requirements  than  those 
who  perform  a  single  role  (i.e.,  with  only  one  type  of  unit  to  control).  For  example,  participants 
who  only  control  WT  do  not  require  information  about  the  localization  and  state  of  households. 
Participants  who  only  control  FF  do  not  need  information  on  the  position  of  lakes.  More 
precisely,  participants  having  a  pure  water-provisioning  role  will  need  to  perform  only  3  out  of 
the  1 1  third-level  subtasks.  Participants  with  only  the  firefighting  role  will  need  to  perform  10  out 
11  tasks.  Participants  with  both  roles  will  need  to  perform  11/11  subtasks.  Information 
requirements  (i.e.,  SA  requirements)  are  therefore  greatly  reduced  when  a  participant  specializes 
in  water-provisioning  but  only  marginally  reduced  when  a  participant  specializes  in  firefighting, 
compared  to  a  participant  performing  both  roles. 
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Situation  assessment 


*  Identify  cell  state 

*  Monitor  wind 
Evaluate  workforce  status 
*  Locate  residential  areas 

*  Locate  lakes 

*  Locate  forests 

*  Evaluate  lire  progression 
Eval.  future  resource  available 
*  Determine  distance  from  fire 

*  Estimate  spreading  speed 

*  Assess  risk  level 

Resource  management 

Elaborate  plan 
Execute  plan 
Monitor  plan  outcomes 
*  2  WT  +  2  FF  units 


Situation  assessment 

*  Identify  cell  state 

*  Monitor  wind 
Evaluate  workforce  status 
*  Locate  residential  areas 

*  Locate  lakes 

*  Locate  forests 

*  Evaluate  lire  progression 
Eval.  future  resource  available 
*  Determine  distance  from  fire 

*  Estimate  spreading  speed 

*  Assess  risk  level 


Resource  management 

Elaborate  plan 
Execute  plan 
Monitor  plan  outcomes 
*  2  WT  +  2  FF  units 


Situation  assessment 

*  Identify  cell  state 

*  Monitor  wind 
Evaluate  workforce  status 
*  Locate  residential  areas 

*  Locate  lakes 

*  Locate  forests 

*  Evaluate  fire  progression 
Eval.  future  resource  available 
*  Determine  distance  from  fire 

*  Estimate  spreading  speed 

*  Assess  risk  level 

Resource  management 

Elaborate  plan 
Execute  plan 
Monitor  plan  outcomes 
*  2  WT  +  2  FF  miits 


*  None 


Prerequisites 


Information  sharing  (Y,  Z) 
Backup  behaviours  (Y,  Z) 
Goal-oriented  coord.  (Y,  Z) 


Information  sharing  (X,  Z) 
Backup  behaviours  (X,  Z) 
Goal-oriented  coord.  (X,  Z) 


Information  sharing  (X,  Y) 
Backup  behaviours  (X,  Y) 
Goal-oriented  coord.  (X,  Y) 


Figure  24:  Task-to-agent  mapping  of  the  multifunctional  team  structure. 
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Situation  assessment 

Evaluate  workforce  status 
*  Locate  lakes 
EvaL  future  resource  available 

Resource  management 

Elaborate  plan 
Execute  plau 
Monitor  plan  outcomes 
*  6  WT  units 


Situation  a  s  ses  s merit 

*  Identify  cell  state 

*  Monitor  wind 
Evaluate  workforce  status 
*  Locate  residential  areas 

*  Locate  forests 

*  Evaluate  fire  progression 
EvaL  future  resource  available 
*  Determine  distance  from  fire 

*  Estimate  spreading  speed 

*  Assess  risk  level 


Resource  management 

Elaborate  plan 
Execute  plan 
Monitor  plan  outcomes 
*  3  FF  units 


Information  sharing  (Y,  Z) 
Backup  behaviours  (Y,  Z) 
Goal-oriented  coord.  (Y ,  Z) 
6  X  resource-oriented 

coordination  (Y,  Z)  * 


Information  sharing  (X,  Z) 
Backup  behaviours  (X,  Z) 
Goal-oriented  coord.  (X,  Z) 
3  X  resource-oriented 

coordination  (X)  * 


Situation  assessment 

*  Identify  ceU  state 

*  Monitor  wind 
Evaluate  workforce  status 
*  Locate  residential  areas 

*  Locate  forests 

*  Evaluate  fire  progression 
EvaL  future  resource  available 
*  Determine  distance  from  fire 

*  Estimate  spreading  speed 

*  Assess  risk  level 


Resource  management 

Elaborate  plan 
Execute  plan 
Monitor  plan  outcomes 
*  3  FF  units 


Information  sharing  (X,  Y) 
Backup  behaviours  (X,  Y) 
Goal-oriented  coord.  (X,  Y) 
3  X  resource-oriented 

coordination  (X)  * 


Figure  25:  Task-to-agent  mapping  of  the  functional  team  structure. 

The  workload  associated  with  situation  assessment  and  tool  interaction  in  the  task-to-agent 
mapping  is  determined  by  allocating  HTA  subtasks  to  participants  according  to  the  roles  they 
perform  and  the  task-tool-role  diagram  shown  in  Figure  23. 
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For  resource  management,  taskwork  depends  on  the  number  of  units  to  control  and  on  the 
subtasks  that  must  be  performed  for  each  unit.  While  there  are  various  resource  management 
subtasks  at  level  3  in  the  HTA,  only  one  of  these  subtasks  can  be  relevant  at  given  time.  For 
example,  within  the  plan  execution  category,  “dispatch  WT  unit  to  FF”  and  “dispatch  WT  unit  to 
lake”  are  mutually  exclusive.  Only  one  of  these  two  actions  is  relevant  for  a  given  WT  unit 
depending  on  the  plan  elaborated  for  that  unit.  Here  is  a  second  example:  When  monitoring  plan 
outcomes  for  a  given  FF  unit,  the  participant  will  either  check  the  firefighting  outcome  or  check  if 
the  refill  is  complete  (not  both),  depending  on  the  plan  executed  for  that  FF  (firefighting  or 
refilling).  We  therefore  assume  that  for  each  unit,  a  participant  must  manage  three  activities  in  a 
cyclic  manner:  one  of  these  activities  is  related  to  plan  elaboration,  another  to  plan  execution  and 
a  third  to  monitoring  plan  outcomes. 

A  numerical  estimate  of  management  workload  can  thus  be  obtained  by  multiplying  the  number 
of  units  controlled  by  this  fixed  number  of  three  subtasks.  For  instance,  if  a  participant  controls 
four  units,  management  workload  is  determined  by  multiplying  4  (units)  by  3  (subtasks),  giving  a 
workload  estimate  of  12. 

To  recapitulate,  the  contents  of  the  taskwork  and  tool  nodes  in  the  task-to-agent  mapping  are 
determined  by  the  task-tool-role  diagram  shown  in  Figure  23.  The  role(s)  of  each  participant 
(firefighting,  water-provisioning,  or  both)  depends  on  the  units  that  he  controls.  Next,  the 
teamwork  node  lists  the  collaboration  processes  that  each  participant  must  perform  and  how  many 
resource-oriented  coordination  processes  are  required  as  a  function  of  the  number  of  units 
controlled.  Teamwork  requirements  were  identified  by  considering  task  dependency  between 
agents  and  finding  the  basic  team  processes  relevant  to  the  simulated  C2  task. 

This  leaves  us  with  the  prerequisite  node  which  identifies  interpersonal  task  dependency.  Clearly, 
a  key  precondition  of  firefighting  is  having  water.  In  functional  teams,  the  performance  of 
Participants  YF  and  ZF  necessarily  depends  on  the  effectiveness  of  Participant  XF  in  providing 
water  to  the  FF  units.  Unlike  the  other  elements  of  the  task-to-agent  mapping,  a  prerequisite  is  not 
seen  as  a  factor  that  contributes  to  an  individual’s  workload,  but  rather  as  a  constraint  on  the 
performance  of  a  team  member  who  depends  on  the  actions  of  a  teammate. 

We  underlined  some  key  elements  in  the  task-to-agent  mapping  and  placed  a  star  besides  them  to 
identify  those  components  that  are  not  constant  for  all  participants.  It  is  what  varies  here  that 
helps  the  most  in  explaining  workload  differences  either  within  a  team  or  between  two  structures. 
There  are  three  distinct  task-to-agent  mappings  amongst  the  six  ones  described  in  Figures  25  and 
26.  The  three  participants  in  the  multifunctional  team  structure  have  the  same  mapping.  In  the 
functional  team  structure,  Participant  X  is  different  from  Participants  Y  and  Z. 

Table  7  summarizes  the  various  sources  of  workload  identified  in  the  task-to-agent  mapping  for 
the  multifunctional  and  the  functional  team  structure.  The  numbers  correspond  to  the 
(unweighted)  workload  associated  with  the  tasks  or  processes  relevant  to  each  dimension.  The 
values  shown  in  Table  7  constitute  the  input  for  the  mathematical  model. 
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Table  7:  Structural  factors  influencing  workload  in  functional  and  multifunctional  teams. 


Participant  & 
structure 

Situation 

assessment 

Resource 

management 

Tool 

interaction 

Teamwork 

Prerequisites 

Multifunctional 

Participant  Xm 

11 

12 

5 

3 

- 

Participant  YM 

11 

12 

5 

3 

- 

Participant  ZM 

11 

12 

5 

3 

- 

Functional 

Participant  XF 

3 

18 

6 

9 

- 

Participant  YF 

10 

9 

5 

6 

Water  from  X 

Participant  ZF 

10 

9 

5 

6 

Water  from  X 

The  task-to-agent  mapping  shows  that  the  main  benefit  of  functional  specialization  comes  from 
having  reduced  SA  requirements,  especially  for  the  water-provisioning  role.  The  main  cost  of 
functional  specialization  is  the  increased  amount  of  teamwork  necessary  to  coordinate  with 
others. 

5.3  Quantitative  modeling 

Here,  we  propose  a  theoretical  account  of  the  factors  that  determine  individual  workload,  we  use 
a  mathematical  function  to  characterize  how  humans  deal  with  increasing  levels  of  workload  and 
how  team  structure  modulates  the  relationship  between  individual  performance  and  team 
effectiveness.  Once  calibrated  on  the  data  from  the  C3Fire  experiment,  the  model  can  serve  as  a 
tool  for  predicting  the  effectiveness  of  other  team  structures. 

The  mathematical  model  uses  the  empirical  data  from  the  C3Fire  experiment  as  a  set  of 
constraints  for  estimating  its  key  parameters.  The  model  does  not  attempt  to  explain  differences 
between  team  having  a  same  structure;  it  only  seeks  to  explain  the  general  difference  between 
structures. 

Table  8  shows  the  average  performance  of  each  team  on  the  four  test  scenarios.  Team 
performance  was  defined  by  the  number  of  fires  put  out  (i.e.,  offensive  aspect)  multiplied  by 
proportion  of  houses  saved  (i.e.,  defensive  aspect).  Performance  was  converted  in  percent  rank 
scores  to  make  comparisons  more  intuitive.  Percent  rank  is  obtained  for  a  given  score  as  follow: 

Number  of  score  under  a  given  value  /  (number  of  score  under  the  value  +  number  of  score  above 
the  value) 

For  instance,  18  scores  are  under  the  score  (61.84)  for  team  #10  (multifunctional)  and  1  is  above. 
Then,  18  /  (18+1)  =  .947. 
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The  average  rank  of  multifunctional  teams  was  76%  compared  to  24%  for  the  functional  teams. 
Team  structure  explains  72%  of  the  variance  in  the  observed  performance  of  the  20  teams 
analyzed.  This  means  that  the  model  can  at  best  explain  72%  of  team  performance  in  the  present 
study. 

Table  8:  Observed  performance  and  percent  rank  for  each  team  in  the  C3Fire  experiment. 


Team  structure 

Team 

Observed  performance 

Percent  rank 

i 

57.75 

0.84 

2 

48.30 

0.63 

3 

63.21 

1.00 

4 

45.23 

0.58 

Multifunctional 

5 

39.10 

0.47 

6 

51.44 

0.68 

7 

56.44 

0.79 

8 

53.95 

0.74 

9 

58.33 

0.89 

10 

61.84 

0.95 

11 

32.80 

0.32 

12 

34.18 

0.37 

13 

38.89 

0.42 

14 

20.75 

0.11 

Functional 

15 

23.23 

0.16 

16 

7.99 

0.00 

17 

16.53 

0.05 

18 

26.75 

0.21 

19 

29.86 

0.26 

20 

44.48 

0.53 

The  mathematical  model  can  be  described  as  a  chain  of  three  mappings  to  explain  team 
performance  (model  output)  based  on  team  structure  (model  input): 

26.  How  team  structure  determines  individual  workload 

27.  How  workload  determines  individual  effectiveness 

28.  How  individual  (and  team)  effectiveness  is  constrained  by  inter-agent  dependency 

The  three  components  of  the  model  are  described  in  turn  below.  Then,  after  calibrating  the  model 
using  empirical  data  from  the  C3Fire  experiment,  we  apply  it  to  new  team  structures  in  order  to 
predict  their  relative  effectiveness. 
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5.3.1  How  team  structure  determines  individual  workload 


The  first  assumption  in  the  model  is  that  the  relative  workload  of  each  individual  depends  on 
taskwork  (situation  assessment  and  resource  management),  teamwork,  and  interaction  with  tools 
required  to  accomplish  the  task.  The  workload  associated  with  each  of  these  factors  was 
characterized  in  the  task-to-agent  mapping  and  then  summarized  in  numerical  form.  The 
mathematical  model  requires  estimating  the  relative  weights  of  these  factors  to  determine  the  total 
workload  of  each  participant  on  a  scale  from  one  to  ten  (like  the  measure  of  perceived  workload). 
Individual  workload  is  computed  by  summing  four  subtypes  of  workload: 

SA  workload  —  Number  of  SA  subtasks  x  weight 
Management  workload  =  Number  of  subtasks  x  number  of  units  x  weight 
Teamwork  workload  =  Number  of  teamwork  processes  X  weight 
Tool  workload  =  Number  of  tools  x  weight 

At  first,  we  assumed  that  the  relative  importance  of  each  of  the  four  workload  factors  would  not 
necessarily  be  equal.  The  modeling  procedure  was  supposed  to  involve  estimating  distinct 
weights  for  each  subtype  of  workload.  However,  preliminary  modeling  results  showed  that  the 
model  successfully  fitted  the  data  with  only  one  weight  parameter  (plus  two  other  free  parameters 
to  define  a  non-linear  function  that  relates  individual  workload  to  effectiveness).  Surprisingly, 
there  was  no  gain  in  allowing  different  weights  values  for  each  factor.  This  leads  to  a  remarkably 
parsimonious  quantitative  model.  Since  this  weight  value  is  the  same  for  all  factors,  it  basically 
plays  the  role  of  a  scaling  parameter  whose  purpose  is  simply  to  resize  the  numerical  workload 
assessment  to  better  fit  perceived  workload. 

The  value  of  the  weight  parameter  was  estimated  by  least-squares  minimization  using  a  quasi¬ 
newton  optimization  algorithm.  The  estimated  value  was  0.249.  The  unweighted  workload  is  thus 
approximately  divided  by  4  in  order  to  fit  subjective  workload.  There  is  an  extremely  strong 
relation  between  perceived  workload  and  the  simple  unweighted  workload  metric  derived  from 
the  agent-to-task  mapping.  In  fact,  the  unweighted  workload  assessment  successfully  explains 
100%  of  the  variance  in  perceived  workload.  This  is  a  remarkable  result  suggesting  that  the 
workload  metric  based  on  the  task-to-agent  mapping  represents  very  accurately  the  actual 
workload  of  individuals.  Table  9  shows  the  observed  and  estimated  workload  for  each  participant 
in  the  multifunctional  and  functional  team  structures. 
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Table  9:  Average  workload  ratings  reported  by  participants  in  the  C3Fire  study  and  model  fits. 


Structure 

Participant 

Perceived 

workload 

Modeled 

workload 

Unweighted 

workload 

Multifunctional 

Participant  XM 

7.73 

7.74 

31 

Participant  YM 

7.73 

7.74 

31 

Participant  ZM 

7.73 

7.74 

31 

Functional 

Participant  XF 

9.00 

8.99 

36 

Participant  YF 

7.50 

7.49 

30 

Participant  ZF 

7.50 

7.49 

30 

Note.  Perceived  workload  was  rated  on  a  scale  from  1  (very  low)  to  10  (very  high).  We  calculated  the 
average  workload  ratings  of  Participants  Xm/Ym/Zm  in  the  multifunctional  structure,  of  Participant  Xp  in 
the  functional  structure  and  of  Participants  Yp/Zp  in  the  functional  structure. 


5.3.2  How  workload  determines  individual  effectiveness 

The  whole  point  of  estimating  the  workload  of  individual  participants  as  a  function  of  team 
structure  is  that  it  can  then  be  related  to  performance.  We  assume  that  the  relationship  between 
workload  and  performance  is  not  just  linear,  but  rather  that  performance  remains  high  as  humans 
compensate  (i.e.,  with  increased  effort  or  adaptive  strategies)  for  increasing  difficulty  and  time 
pressure,  then  rapidly  drops  past  a  point  of  overload,  and  stabilizes  at  some  minimum  (see  [94]). 
We  assume  that  this  relationship  follows  an  inverse  sigmoid  function.  Figure  26  illustrates  a 
generic  version  of  this  function. 


Figure  26:  Typical  ( reverse )  sigmoid  function  (n  =  5,  k  -  5,  max  -  1). 
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This  function  can  drop  more  or  less  steeply  or  at  different  points  on  the  x-axis  depending  on  its 
parameters.  The  equation  for  the  reverse  sigmoid  is: 

y  =  max  •  kn  /  (kn  +  xn) 

where  “y”  corresponds  to  the  output  of  the  function:  individual  effectiveness.  The  max  parameter 
corresponds  to  the  maximum  value  of  y.  The  k  parameter  is  the  “half-maximum”  of  the  curve  and 
has  the  following  meaning:  when  x  =  k,  y  =  1/2  •  max.  The  “w”  parameter  controls  how  steeply 
the  curve  falls.  When  modeling  the  data  from  the  C3Fire  experiment,  the  exact  shape  of  the 
function  is  estimated  by  the  model  using  least-squares  minimization.  The  function  has  one  fixed 
and  2  free  parameters: 

♦  max  is  set  to  one  (i.e.,  the  optimal  effectiveness) 

♦  the  estimated  k  value  is  8.39 

♦  n  is  estimated  to  be  14.20 

Figure  27  illustrates  the  functional  relationship  between  individual  workload  and  effectiveness 
estimated  by  the  model. 


Figure  27:  Estimated  relation  between  workload  and  effectiveness. 

Table  10  shows  the  predicted  effectiveness  of  each  team  member  as  a  function  of  its  workload. 
Note  that  participant  XF  is  clearly  overloaded  compared  to  other  participants. 
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Table  10:  Individual  workload  and  predicted  effectiveness  as  a  function  of  team  structure. 


Structure 

Participant 

Effectiveness 

Workload 

Multifunctional 

Participant  XM 

0.76 

7.74 

Participant  YM 

0.76 

7.74 

Participant  ZM 

0.76 

7.74 

Functional 

Participant  XF 

0.27 

8.99 

Participant  YF 

0.83 

7.49 

Participant  ZF 

0.83 

7.49 

5.3.3  How  individual  effectiveness  is  constrained  by  interpersonal 
dependency 

Despite  the  very  low  effectiveness  predicted  for  Participant  XF,  the  model  still  cannot  adequately 
account  for  the  large  gap  between  the  average  performance  of  functional  and  multifunctional 
teams.  A  key  structural  difference  remains  to  be  added.  There  is  notion  of  dependency  between 
the  participants  YF  and  ZF  and  participant  XF.  The  first  two  participants  must  accomplish  two 
distinct  goals.  One  is  the  offensive  aspect  of  the  game  (closing  out  the  fires)  and  the  other  is  the 
defensive  one  (saving  the  houses).  The  goal  of  participant  XF  is  only  to  refill  the  other  two 
participants’  units  with  water  when  required.  Consequently,  the  goal  fulfillment  of  the 
participants  YF  and  ZF  is  totally  dependent  of  the  goal  fulfillment  of  participant  XF.  As  a  result,  it 
becomes  a  negative  constraint  on  these  participants  as  per  the  following  formalization. 

This  constraint  on  the  performance  of  Participants  YF  and  ZF  is  operationalized  by  multiplying 
their  effectiveness  with  that  of  Participant  XF.  The  actual  performance  of  Participants  YF  and  ZF  is 
0.83  (their  own  effectiveness)  *  0.27  (the  effectiveness  of  Participant  XF),  for  a  result  of  0.23. 
Team  performance  is  computed  to  by  averaging  the  (constrained)  performance  of  all  three  team 
members.  Table  11  shows  the  main  modeling  results:  estimated  workload,  constrained 
effectiveness  and  predicted  team  performance. 
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Table  IT.  Workload  and  predicted  effectiveness  as  a  function  of  team  structure. 


Structure 

Predicted  team 
performance 

Constrained 

effectiveness 

Estimated 

workload 

Participant 

0.76 

7.74 

Participant  X 

Multifunctional 

0.76 

0.76 

7.74 

Participant  Y 

0.76 

7.74 

Participant  Z 

0.27 

8.99 

Participant  X 

Functional 

0.24 

0.23* 

7.49 

Participant  Y 

0.23* 

7.49 

Participant  Z 

*  The  unconstrained  effectiveness  of  Participants  Yp  and  Zp  was  0.83.  This  value  was  multiplied  by  the 
effectiveness  of  Participant  Xp  on  which  they  depend  (0.27),  giving  a  constrained  effectiveness  of  0.23. 


The  average  performance  of  multifunctional  teams  in  the  C3Fire  experiment  was  0.76.  The 
average  performance  of  functional  teams  was  0.24.  The  mathematical  model  therefore 
successfully  fitted  the  observed  workload  ratings  and  the  average  performance  of  each  team 
structure  as  well  as  was  theoretically  possible.  Although  the  model  does  not  account  for 
individual  team  variability  (which  depends  on  other  factors  than  team  structure)  it  successfully 
explains  72%  of  the  variability  in  the  observed  performance  of  the  20  teams  tested.  Clearly,  team 
structure  is  a  determinant  factor  of  team  effectiveness.  The  mathematical  model  provides  a  good 
theoretical  explanation  of  how  structural  factors  may  have  influenced  team  effectiveness. 

5.4  Application  to  new  structures 

According  to  Navarro  and  Lee  [95]  a  good  model  should  1)  provide  accurate  descriptions  of  the 
available  data;  2)  confer  meaning  or  offer  substantive  insight  into  the  phenomena  being 
investigated;  and  3)  provide  predictions  and  generalize  to  new  or  different  situations  where  data 
are  not  available.  The  mathematical  model  developed  here  -  now  that  it  has  been  successfully 
calibrated  on  empirical  data  -  can  serve  as  a  tool  to  extrapolate  how  teams  should  perform  on  the 
same  task  when  organized  according  to  other  possible  structures.  Four  candidate  team  structures 
came  to  mind: 

♦  An  alternate  form  of  the  functional  organization 

♦  A  hybrid  team  structure  (part  functional,  part  multifunctional) 

♦  A  four-person  functional  team  structure 

♦  A  six  person  multifunctional  team  structure 

Unit  allocation  is: 

-  [X  =  6FF  //  Y  =  3WT  //  Z  =  3WT]  in  the  alternate  functional  structure 

-  [X  =  2FF-2WT  //  Y  =  4WT  //  Z  =  4FF]  in  the  hybrid  structure 
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_  [W  =  3WT  //  X  =  3WT  //  Y  =  3FF  //  Z  =  3FF]  in  the  4-functional  structure 
-  [1FF-1WT  each]  in  the  6-multifunctional  structure 

We  performed  a  new  task-to-agent  mapping  for  each  of  these  four  test  structures.  Table  12 
summarizes  the  key  structural  factors  that  define  the  total  workload  of  each  participant. 

Table  12:  Key  structural  factors  influencing  individual  workload  in  the  four  test  structures. 


Participant  & 
structure 

Situation 

assessment 

Resource 

management 

Tool 

interaction 

Teamwork 

Prerequisites 

Alt.  functional 

Participant  X 

10 

18 

5 

9 

Water  from  Y-Z 

Participant  Y 

3 

9 

6 

6 

- 

Participant  Z 

3 

9 

6 

6 

- 

Hybrid 

Participant  X 

11 

12 

5 

3 

- 

Participant  Y 

3 

12 

6 

7 

- 

Participant  Z 

10 

12 

5 

7 

Water  from  Y 

4-functional 

Participant  W 

3 

9 

6 

6 

- 

Participant  X 

3 

9 

6 

6 

- 

Participant  Y 

10 

9 

5 

6 

Water  from  W 

Participant  Z 

10 

9 

5 

6 

Water  from  X 

6-multifunctional 

Participant  U 

11 

6 

5 

3 

- 

Participant  V 

11 

6 

5 

3 

- 

Participant  W 

11 

6 

5 

3 

- 

Participant  X 

11 

6 

5 

3 

- 

Participant  Y 

11 

6 

5 

3 

- 

Participant  Z 

11 

6 

5 

3 

- 

We  applied  the  mathematical  model  to  each  of  the  four  test  structures  in  order  to  infer  their 
relative  effectiveness.  Predictions  were  obtained  using  the  parameters  that  were  estimated  earlier 
using  results  from  the  functional  and  multifunctional  teams  in  the  C3Fire  experiment.  Table  13 
shows  the  predicted  effectiveness  for  each  of  these  team  structures. 
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Table  13:  Application  of  the  model  as  a  tool  for  estimating  the  effectiveness  of  four  team 

structures. 


Structure 

Predicted  team 
performance* 

Constrained 

effectiveness 

Estimated 

workload 

Participant 

Alternate 

0.69 

0.07 

10.00** 

Participant  X 

functional 

0.99 

5.99 

Participant  Y 

0.99 

5.99 

Participant  Z 

Hybrid 

0.70 

0.76 

7.74 

Participant  X 

0.93 

6.99 

Participant  Y 

0.42 

8.49 

Participant  Z 

4-person 

0.91 

0.99 

5.99 

Participant  W 

functional 

0.99 

5.99 

Participant  X 

0.83 

7.49 

Participant  Y 

0.83 

7.49 

Participant  Z 

6-person 

0.99 

0.99 

6.24 

Participant  U 

multifunctional 

0.99 

6.24 

Participant  V 

0.99 

6.24 

Participant  W 

0.99 

6.24 

Participant  X 

0.99 

6.24 

Participant  Y 

0.99 

6.24 

Participant  Z 

*  Performance  (in  percent  rank)  relative  to  the  performance  of  the  20  teams  in  the  C3Fire  experiment. 
**  Workload  is  assumed  to  be  maximal  at  10. 


♦  Alternate  functional  team.  In  an  alternate  functional  division  of  labour,  Participant  X 
has  6  FF,  Participant  Y  has  3  WT,  and  Participant  Z  has  3  WT.  Participant  X 
depends  on  both  of  his  teammates  for  water  and  he  has  a  maximal  workload  (the 
estimated  value  would  be  higher  but  it  has  been  limited  to  10).  Despite  his  low 
effectiveness,  Participant  X  seldom  lacks  water  due  to  the  high  effectiveness  of 
Participants  Y  and  Z  in  replenishing  the  water  reserves  of  his  units. 

♦  Hybrid  team.  It  is  possible  to  mix  a  functional  and  a  multifunctional  task  allocation 
within  a  same  team.  In  this  hybrid  structure,  Participant  X  is  multifunctional.  He 
controls  4  FF  and  4  WT  units.  Participant  Y  controls  2  WT  units  and  Participant  Z 
control  2  FF  units.  The  model  predicts  that  Participant  Z  should  be  overloaded  and 
that  this  will  reduced  overall  team  performance.  This  hybrid  structure,  while  not 
optimal  in  terms  of  predicted  effectiveness,  may  benefit  from  a  greater  adaptability 
to  various  situations. 
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♦  4-Person  functional  team.  An  interesting  possibility  offered  by  the  model  is  to 
explore  how  increasing  the  number  of  team  members  may  impact  the  effectiveness 
of  a  given  team  structure.  While  a  functional  division  of  labor  in  a  three-person  team 
resulted  in  an  overloaded  individual,  the  same  may  not  be  true  for  a  four-person 
team.  In  this  team  configuration,  Participants  W  and  X  control  3  WT  each,  while 
Participants  Y  and  Z  control  3  FF  each.  The  model  predicts  that  a  4-person 
functional  team  should  perform  markedly  better  than  a  three-person  multifunctional 
team  (91  vs  76  percent  rank). 

♦  6-Person  multifunctional  team.  Another  potential  use  for  the  model  is  to  determine 
how  many  people  are  required  to  perform  the  task  as  well  as  possible.  The  model 
can  be  used  to  predict  at  which  point  increasing  the  number  of  team  members  no 
longer  provides  a  benefit.  For  instance,  when  six  participants  each  control  1  FF  and 
1  WT,  the  model  predicts  that  workload  will  be  sufficiently  low  so  that  effectiveness 
is  nearly  optimal. 

In  summary,  the  mathematical  model  predicts  the  following  ordering  of  team  effectiveness  as  a 
function  of  team  structure: 

♦  6-Person  multifunctional  (99%) 

♦  4-Person  functional  (91%) 

♦  Multifunctional  (76%)) 

♦  Hybrid  (70%) 

♦  Alternate  functional  (69%) 

♦  Functional  (24%) 
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6  General  discussion 


The  present  work  has  shown  that  combining  experimental  measurement  using  a  microworld, 
hierarchical  task  analysis  and  mathematical  modeling  can  provide  a  systematic  method  for 
estimating  the  costs/benefits  of  different  team  architectures.  Current  results  suggest  that  the 
approach  has  great  potential.  By  providing  clear  quantitative  predictions,  the  model  is  highly 
refutable  and  can  therefore  be  improved  if  some  predictions  do  not  turn  out  to  be  correct.  If 
necessary,  the  model  could  also  be  calibrated  on  more  than  just  two  team  structures  to  increase 
the  reliability  of  its  predictions.  Thanks  to  its  great  simplicity,  this  approach  has  shown  that  it  can 
obtain  stable  estimates  of  its  three  adjustable  parameters  (taskwork,  teamwork  and  tool 
interaction)  without  requiring  excessive  empirical  data.  This  method  therefore  overcomes  a 
fundamental  limitation  of  computational  GOMS  models  and  complex  cognitive  architectures  [56] 
discussed  earlier  in  this  report.  Such  models  attempt  to  characterize  processes  and  subtasks  with 
such  detail  that  estimating  the  numerous  unknown  parameters  appropriately  would  require  an 
excessively  large  number  of  experimental  manipulations  and  measurements.  This  problem  is 
similar  to  the  notion  of  “sample  complexity”  in  machine  learning:  the  more  complex  the  model, 
the  more  examples  (i.e.,  constraints)  it  requires  in  order  to  make  reliable  predictions.  This 
problem  applies  as  well  to  the  TOD  and  the  IPME  methodologies  described  in  Freeman  et  al. 
[53].  These  quantitative  modeling  approaches  involve  a  model  calibration  phase  based  on 
empirical  data  in  order  to  generate  predictions,  just  like  the  method  proposed  here. 

A  common  element  to  the  mathematical  model  proposed  here,  the  computational  GOMS  model, 
the  TOD  methodology  and  the  IPME  framework  is  that  the  concept  of  workload  is  very 
important.  In  fact,  the  TOD  methodology  proposes  two  alternative  ways  to  model  the  effects  of 
workload  [55].  Figure  28  shows  the  Gaussian  workload  function  and  the  inverted  quadratic 
workload  function.  Note  that  the  latter  function  is  remarkably  similar  to  inverse  sigmoid  proposed 
in  the  present  report. 


Figure  28:  Alternate  forms  for  the  effect  of  workload  on  operator  accuracy 

(from  [55]). 

Despite  the  similar  approach,  we  argue  that  the  computational  GOMS,  TOD  and  IPME  models 
are  unnecessarily  complex  and  that  this  can  seriously  limit  their  reliability  when  calibrated  on 
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limited  experimental  data.  The  time,  effort,  expertise  and  cost  required  to  develop  and  apply  these 
models  may  be  greater  than  necessary.  For  example,  Levchuk  et  al.  [55]  model  task  effectiveness 
using  many  variables  related  to  individual  characteristics  such  as  initial  incompetence,  learning 
and  memory  (retention  of  learning).  These  free  parameters  are  not  essential  to  predict  the 
effectiveness  of  various  team  structures  and  they  may  provide  too  much  flexibility  to  these 
models,  thus  increasing  the  risk  of  noise  overfitting  [57]. 

The  method  taken  here  to  estimate  the  costs/benefits  of  teamwork  takes  an  approach  that  is 
somewhat  similar  to  the  EAST  methodology  (Event  Analysis  of  Systemic  Teamwork).  EAST 
combines  a  number  of  human  factors  research  methods  to  study  team  activity  in  a  C4I  work 
context  (command,  control,  communications,  computers  and  intelligence),  namely: 

♦  Hierarchical  task  analysis 

♦  Observation  (empirical  data) 

♦  Coordination  demands  analysis 

♦  Communications  usage  diagrams 

♦  Social  network  analysis 

♦  Operation  sequence  diagram 

♦  Critical  decision  method 

♦  Propositional  networks 

Note  that  the  two  first  methods  in  this  list  (HTA  and  observation/measurement)  correspond  to 
those  used  in  the  present  study.  The  task-to-agent  mapping  accomplishes  a  purpose  similar  to 
coordination  demands  analysis  (defining  taskwork  and  teamwork). 

“EAST  provides  an  assessment  of  agent  roles  within  the  network,  a  description 

of  the  activity  including  the  flow  of  information,  the  component  tasks, 

communication  between  agents  and  the  operational  loading  of  each  agent. 

Coordination  between  agents  is  also  rated  and  the  knowledge  required 

throughout  the  task  under  analysis  is  defined”  [96]. 

The  EAST  methodology  is  an  assembly  of  distinct  techniques  capable  of  providing  multiple 
views  on  team  activity.  The  descriptive  capabilities  of  this  approach  are  extensive.  Stanton  et  al. 
[96]  note  however  that  the  method  it  is  very  time  consuming  because  it  is  so  exhaustive. 

The  approach  developed  here  does  not  apply  such  a  wide  range  of  methods  of  analysis,  yet  its 
greater  simplicity  and  focus  have  allowed  it  to  produce  a  mathematical  model  capable  of 
predicting  team  effectiveness  as  a  function  of  team  structure.  The  main  novelty  of  the  present 
method  lies  in  the  effort  to  integrate  results  from  each  method  to  produce  a  well-defined 
theoretical  framework  capable  of  making  a  priori  predictions. 
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The  present  project  has  proposed  a  new  framework  for  estimating  the  costs  and  benefits  of 
teamwork  because  no  prior  framework  was  capable  of  providing  reliable  quantitative  estimates  of 
team  effectiveness  based  on  minimal  data  collection. 

While  the  FRAM  analysis  was  a  major  inspiration  for  the  task-to-agent  mapping  performed  here, 
its  objective  is  not  to  predict  the  effects  of  various  team  structures  on  team  effectiveness. 
Hollnagel  [83]  and  Sawaragi,  Horiguchil  &  Hinal  [97],  used  FRAM  for  safety  analysis.  Woltjier 
et  al.  [84]  applied  the  FRAM  method  to  inform  the  design  of  information  technology  to  be  used  in 
network-based  command-and-control  centers.  Our  framework  draws  only  from  “Step  1”  in  the 
FRAM  analysis,  which  identifies  the  functions  and  constraints  on  the  execution  of  a  task.  Like  a 
HTA,  the  first  part  of  the  FRAM  analysis  is  conducted  by  (or  in  consultation  with)  a  domain 
expert.  The  key  difference  between  the  two  approaches  is  that  the  FRAM  modules  are  centered 
on  functions  aimed  at  process  control,  while  the  task-to-agent  mapping  is  centered  on  each  agent 
(i.e.  each  agent  constitutes  a  module).  Later  steps  in  the  FRAM  analysis  are  concerned  with 
identifying  the  variables  that  change  during  task  execution  and  the  constraints  on  these  variables. 
In  principle,  a  different  FRAM  analysis  could  be  performed  for  each  possible  team  structure  in 
order  to  predict  the  effectiveness  of  various  team  structures.  However,  very  few  of  the  key  factors 
or  predictors  in  the  analysis  (called  common  performance  conditions)  are  in  fact  related  to  team 
structure: 

♦  Adequacy  of  organization  (related  to  structure  but  circular) 

♦  Working  conditions 

♦  Adequacy  of  interface  and  operational  support 

♦  Availability  of  procedures  /  plans 

♦  Number  of  simultaneous  goals  (related  to  structure) 

♦  Available  time 

♦  Circadian  rhythm 

♦  Adequacy  of  training  and  experience 

♦  Crew  collaboration  quality 

♦  Communication  (related  to  structure) 

♦  Availability  of  resources 

The  new  framework  proposed  here  is  therefore  more  appropriate  for  predicting  team  effectiveness 
as  a  function  of  team  structure  than  the  FRAM  analysis  because  it  relies  specifically  on  factors 
that  vary  according  to  team  structure  (taskwork,  teamwork,  tool  interaction,  interpersonal  task 
dependency). 

The  structural  factors  identified  in  the  theoretical  model  may  provide  further  insights  on  how  to 
improve  performance  in  functional  teams.  For  example  one  way  to  improve  the  effectiveness  of 
functional  teams  could  be  to  provide  team  members  with  direct  access  to  the  key  information 
necessary  for  resource-oriented  coordination  (e.g.,  showing  to  Participant  XF  the  position  of  FF 
units  on  the  map  to  facilitate  water-provisioning).  Automated  information  sharing  (of  data 
relevant  to  the  collaboration  process)  should  help  reduce  individual  workload  and  avoid  the 
disruptive  effects  interruptions  associated  with  information  requests. 
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The  present  work  did  not  take  into  account  team  factors  such  as  trust,  team  cohesion  and  team 
members  experience  working  together.  Yet,  incorporating  these  factors  would  allow  modelling 
individual  differences  across  teams.  Furthermore,  teamwork  requirements  may  evolve  throughout 
the  existence  of  the  team.  Thus,  team  mates  working  for  an  extended  period  of  time  together  in  a 
functional  structure  could  have  shown  better  results  than  in  multifunctional  one.  This  question 
could  make  the  object  of  future  works. 

As  a  final  remark,  we  should  add  that  future  research  would  benefit  from  a  more  detailed 
workload  questionnaire  since  the  theoretical  model  relies  heavily  on  that  notion.  Despite  the  great 
complexity  and  context  dependence  of  team  functioning,  this  new  approach  to  team  design  shows 
that  estimating  the  costs  and  benefits  of  teamwork  can  be  performed  in  a  systematic  and  tractable 
way.  The  approach  identifies  how  to  balance  the  workload  of  C2  teams  and  can  determine  how 
many  team  members  are  necessary  to  optimize  team  effectiveness  in  complex  and  dynamic 
situations. 
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7  Conclusion 


The  next  sections  summarize  major  contributions  from  this  research  project. 

7.1  Identification  of  team  processes 

One  major  impact  of  this  project  is  to  demonstrate  that  the  relationship  between  key  determinants 
of  teamwork  and  team  effectiveness  varies  as  a  function  of  team  structure.  This  demonstration 
comes  from  the  comparison  of  the  coefficients  found  in  two  multiple  regression  models  (for 
multifunctional  and  functional  teams)  relating  team  performance  to  a  number  of  predictors  of 
team  effectiveness.  Key  similarities  and  differences  were  observed,  suggesting  that  some  aspects 
of  team  functioning  can  markedly  change  as  a  function  of  team  structure.  Since  different  team 
structures  may  rely  upon  different  team  processes,  it  would  seem  inadequate  for  future  research  to 
employ  a  general  model  of  team  performance  that  uses  a  fixed  set  of  standard  team  processes  as 
predictor  variables.  The  role  of  these  processes  is  not  the  same  from  one  team  structure  to 
another,  and  so  such  processes  are  an  insufficient  way  to  predict  the  performance  of  different 
team  structures.  The  findings  presented  in  this  report  suggest  that  teamwork  requirements  and  the 
importance  of  various  team  factors  in  general  may  vary  as  a  function  of  team  structure. 
Obviously,  communication  and  coordination  are  key  components  of  teamwork,  but  their  relative 
importance  is  affected  by  the  manner  in  which  the  team  is  configured.  For  instance,  results 
showed  that  communication  was  positively  correlated  with  team  performance  in  functional  teams, 
but  negatively  correlated  in  multi-functional  ones. 

Depending  on  the  team  structure,  each  team  member  has  to  perform  particular  tasks  associated 
with  his  or  her  role  within  the  team.  Team  members  also  need  to  coordinate  their  interdependent 
activities,  share  information  and  make  collaborative  decisions  on  how  to  deal  with  the  situation. 
Yet  many  social  and  cognitive  factors  interact  to  determine  team  effectiveness  under  nominal 
situation  and  in  off-nominal  situations  in  which  resilience  and  adaptation  (i.e.,  team  agility)  are 
critical.  Are  optimally  effective  teams  also  the  most  resilient  and  adaptive,  or  is  there  a  trade-off 
in  these  properties.  It  could  be  argued  that  resilience  needs  redundancy,  which  is  typically  not  the 
way  to  maximise  effectiveness.  There  remains  a  lack  of  consensus  regarding  the  delimitation  and 
conceptualization  of  processes  and  behaviours  critical  for  team  effectiveness  and  agility  in  the 
literature  on  teamwork.  Consequently,  there  is  a  growing  need  to  understand  teamwork  and  the 
processes  or  factors  that  affect  both  team  performance  and  agility  as  well  as  how  these  two 
properties  interact.  Furthermore,  results  from  this  research  project  highlight  the  importance  of 
considering  that  the  impact  of  many  of  the  relevant  processes  and  factors  may  be  structure- 
dependent. 

Our  results  showed  that  team  performance  as  a  function  of  team  structure  can  be  predicted  by 
mathematical  models  that  use  factors  such  as  taskwork,  teamwork,  tool  interaction  and 
interpersonal  task  dependency.  The  possibility  to  use  such  models  in  order  to  identify  the  optimal 
type  of  team  structure  and  the  optimal  number  of  team  members  is  promising. 
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7.2  Workload:  a  key  predictor  of  team  performance 

Individual  workload  and  interpersonal  dependency  lie  at  the  heart  of  the  mathematical  model 
described  in  this  report.  Workload  is  considered  as  a  key  predictor  of  the  team  performance.  The 
mathematical  model  can  be  described  as  a  chain  of  three  mappings  to  explain  team  performance 
(model  output)  based  on  team  structure  (model  input): 

29.  How  team  structure  determines  individual  workload 

30.  How  workload  determines  individual  effectiveness 

31.  How  individual  (and  team)  effectiveness  is  constrained  by  inter-agent  dependency 

Consequently,  a  critical  phase  in  the  development  of  the  mathematical  model  was  to  define  the 
concept  of  workload.  The  concept  of  workload  was  defined  by  considering  various  elements 
shown  in  the  task-to-agent  mapping: 

32.  the  various  subtasks  assigned  to  the  participant; 

33.  the  interaction  with  tools ; 

34.  the  teamwork  necessary  to  accomplish  individual  and  team  goals. 

Such  decomposition  of  workload  was  key  to  the  modeling  approach.  The  correlation  between  the 
level  of  workload  perceived  by  the  participants  and  the  results  from  the  model  is  astonishing.  This 
remarkable  result  suggests  that  the  workload  metric  based  on  the  task-to-agent  mapping 
represents  accurately  the  actual  workload  of  individuals. 

7.3  Using  the  model  to  identify  optimal  team  structure  and 
size 

In  this  project,  an  experiment  demonstrated  that  using  different  team  structures  led  to 
different  team  performance.  We  performed  a  task  analysis  and  developed  a  structural  model  (a 
task-to-agent  mapping)  that  represented  team  workload  and  then  developed  a  mathematical  model 
of  team  effectiveness.  The  formal  model  was  very  successful  in  accounting  for  the  experimental 
results.  The  mathematical  model  displays  two  different  interesting  capabilities: 

First,  the  model  can  serve  as  a  tool  to  extrapolate  how  teams  should  perform  on  the  same  task 
when  organized  according  to  other  possible  structures.  This  model  capability  is  very  interesting 
for  the  identification  of  optimal  team  structure  in  C2  environment. 

Another  interesting  possibility  offered  by  the  model  is  to  explore  how  increasing  the  number  of 
team  members  may  impact  the  effectiveness  of  a  given  team  structure.  For  instance,  the  model 
predicted  that  a  4-person  functional  team  should  perform  markedly  better  than  a  three-person 
functional  team  (91  vs  24  percent  rank).  This  result  suggests  that  adding  one  person  to  the  team 
may  result  in  an  important  increase  in  the  team  performance. 
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7.4  Future  works 


This  report  presents  results  of  an  empirical  test  and  a  mathematical  model  of  team  performance 
with  regard  to  the  team  structure.  The  model  displays  an  impressive  capacity  to  replicate  the 
results  observed  from  the  20  three-person  teams.  Thanks  to  its  great  simplicity,  this  approach  has 
shown  that  it  can  obtain  stable  estimates  of  its  three  adjustable  parameters  without  requiring 
excessive  empirical  data.  From  this  model,  it  was  possible  to  predict  the  performance  of  four 
other  fictional  teams.  Three  phases  should  follow  the  work  initiated  in  this  project: 

Phase  1 :  The  comparison  of  the  results  predicted  by  the  model  with  the  four  different  team 
structures  (Six-Person  multifunctional;  four-Person  functional;  Hybrid;  and  Alternate  functional) 
with  the  performance  of  teams  using  these  different  structures.  This  would  provide  a  test  of  the 
specific  assumptions  proposed  in  the  model  and  help  improve  its  reliability. 

Phase  2:  The  replication  of  the  results  presented  in  this  report  with  a  different  microworld  in 
order  to  generalize  the  findings  and  increase  the  ecological  validity  of  the  approach. 

Phase  3:  The  validation  of  the  capacity  of  the  model  to  assist  team  design  by  testing  its 
predictions  of  performance  for  real  C2  teams  in  the  ecological  context  of  a  rapid  response 
planning  exercise  or  integrated  C2  experimentation. 
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published  thesaurus,  e.g.  Thesaurus  of  Engineering  and  Scientific  Terms  (TEST)  and  that  thesaurus  identified.  If  it  is  not  possible  to  select 
indexing  terms  which  are  Unclassified,  the  classification  of  each  should  be  indicated  as  with  the  title.) 
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Computational  Modelling;  Performance  Prediction 
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